You do not need a paid subscription to run a capable AI model. A free, open-source tool called Ollama lets you download models like Llama 3.2, Mistral and Google’s Gemma and run them directly on your own laptop or desktop — no monthly fee, no data leaving your machine, and no internet connection needed once the model is downloaded.
Ollama has become one of the most-starred artificial-intelligence projects on GitHub, with a community measured in the hundreds of thousands. The appeal is simple: it turns the messy business of running a large language model locally into two or three commands. Below is exactly how to set it up, which model to choose, and what it really costs you.
What Ollama actually does
A large language model is just a very big file of numbers. Normally you rent access to one through a cloud service and pay per use. Ollama flips that around. It downloads the model file to your computer and gives you a small program that feeds your questions in and streams the answers back. Everything happens on your hardware. Your prompts never touch a company server, which matters if you handle private notes, client data, or anything you would rather not upload.
It runs on macOS, Windows and Linux, and it works on ordinary consumer hardware. A recent MacBook or a gaming PC with a decent graphics card runs mid-size models comfortably. A plain office laptop can still run the smaller ones.
How to install Ollama, step by step
macOS: Go to ollama.com, download the app, and drag it into your Applications folder. Open it once so it can install its command-line tool.
Windows: Download the Windows installer from ollama.com and run it. It sets up a background service automatically.
Linux: Open a terminal and run one line:
curl -fsSL https://ollama.com/install.sh | shThat is the whole installation. To confirm it worked, open a terminal (or Command Prompt on Windows) and type ollama --version. If it prints a version number, you are ready.
Run your first model
One command downloads a model and starts a chat with it. Try Meta’s Llama 3.2, a small, fast model that fits on most machines:
ollama run llama3.2The first time, it downloads the model — a few gigabytes, so give it a minute on a normal connection. After that it loads instantly and works offline. When the prompt appears, just type a question and press Enter. Type /bye to leave the chat. The model stays on your disk, so the next ollama run llama3.2 starts in seconds.
Which model should you pick?
Model size is measured in billions of parameters (the “3b” or “7b” in the name) and roughly tracks how much memory you need:
- llama3.2 (3B): light and quick. Runs on 8GB of RAM. Good for everyday questions, drafting and summaries.
- mistral (7B): a strong all-rounder. Wants about 16GB of RAM for smooth use.
- gemma2 (9B) or llama3.1 (8B): more capable reasoning, best with a dedicated graphics card.
- phi3 (mini): tiny and surprisingly sharp; a good pick for older or low-memory machines.
Swap the name in the run command to try any of them, for example ollama run mistral. Start small. If answers come fast and your machine stays responsive, step up to a larger model.
The free-lane tip: turn it into an API
Once Ollama is running, it quietly exposes a local web endpoint at http://localhost:11434. That means other free apps can talk to it as if it were a paid AI service — without the bill. Point an open-source chat interface like Open WebUI at that address and you get a clean, browser-based assistant running entirely on your own computer. Developers can call the same endpoint from their own scripts. This is how people build private, no-cost AI tools that never send a single word to an outside company.
Is it really free? What it actually costs you
The software and the models are free to download and use. What you pay instead is hardware and electricity. A larger model runs slower on a weak machine, and heavy use warms up your laptop and draws power. For most people that trade is easily worth it: unlimited use, full privacy, and no subscription. If your computer is old, stick to the smaller models and you will still get real work done.
The bottom line
Running your own AI model used to be a specialist task. Ollama has reduced it to a download and one command. If you have been paying monthly for a chatbot you use lightly, an afternoon with Ollama may replace it for nothing — and keep your data on your own machine while it does.
