Blog · Getting Started · September 17, 2026 · 7 min read

Getting Started with Ollama: Install and Run AI Models on Your Own Machine

Ollama is the easiest way to run open-source AI models on your own computer. It bundles the model runtime, weight management, and an OpenAI-compatible API server into a single tool — no Python environments, no CUDA setup, no Docker. If you have ever wanted a private, offline, zero-cost chatbot that never sends your data to a cloud provider, this is the fastest path there.

What you need

• Operating system: Windows 10 or newer, macOS 11+, or a modern Linux distribution.
• RAM/VRAM: 8 GB of memory is enough for small models (1–3B). For popular 7–8B models, 8 GB of VRAM (or 16 GB of system RAM if running on CPU) gives a comfortable experience.
• A GPU (optional): NVIDIA or AMD cards dramatically speed things up, but Ollama runs fine on CPU — just slower. Apple Silicon Macs use unified memory and work excellently.
• Disk space: models are large files — a 7B model is roughly 4–5 GB on disk.

Step 1: Install Ollama

Download the installer for your platform from ollama.com/download, or use a package manager:

Windows: download and run OllamaSetup.exe. Ollama runs in the system tray and starts automatically.

macOS: download and drag the app to Applications, or with Homebrew:

brew install ollama

Linux: one command installs everything:

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Verify the installation

Open a terminal (PowerShell on Windows) and check the version:

ollama --version

If the command is not found on Windows, close and reopen your terminal so the updated PATH is picked up.

Step 3: Pull and run your first model

Grab a popular 8B model — a great balance of quality and hardware requirements:

ollama pull llama3.1:8b

Then start chatting with it interactively:

ollama run llama3.1:8b

Type a message and press Enter. Useful commands inside the chat: /help shows all options, /clear resets the conversation, and /bye exits. You can also skip the pull step entirely — ollama run <model> downloads the model first if it is missing.

Other great starter models: qwen2.5:7b (strong multilingual and coding), gemma2:9b (efficient and capable), and deepseek-r1:8b (shows its reasoning steps before answering).

Step 4: Manage your models

Step 5: Use the built-in API

Ollama serves an API at http://localhost:11434 the moment it is running. Quick test with curl:

curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1:8b",
  "prompt": "Why is the sky blue?",
  "stream": false
}'

Better yet, Ollama speaks the OpenAI API format, so almost every AI tool and library works with it by changing one URL:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

response = client.chat.completions.create(
    model="llama3.1:8b",
    messages=[{"role": "user", "content": "Hello, local world!"}],
)
print(response.choices[0].message.content)

Which model fits my hardware?

Ollama downloads models as GGUF files and defaults to roughly 4-bit quantization (Q4), which keeps quality high while cutting memory use to about 0.6 GB per billion parameters, plus 1–2 GB of overhead for context. Rule-of-thumb table:

For a precise answer, use the compatibility panel on our home page: select your GPU and check the Q4 row — that matches what Ollama actually downloads. The tokens/second estimate there also closely matches what Ollama delivers on the same card.

Troubleshooting

• “Connection refused” / ollama not responding: the server process is not running. Start it with ollama serve and check for errors.
• Very slow responses: you are likely running on CPU. Try a smaller model, or verify your GPU drivers are up to date — Ollama logs the detected GPU at startup.
• Out of memory: pick a smaller model or a smaller context. On Windows you can also move the model storage (default C:\Users\<you>\.ollama\models) to another drive with the OLLAMA_MODELS environment variable.
• Long context needs: Ollama defaults to a modest context window. Raise it per request with "options": { "num_ctx": 16384 } — and remember every extra context token costs VRAM (our compatibility slider shows exactly how much).

Next steps

• Create custom assistants with a Modelfile — a tiny config that sets a system prompt, temperature, and a base model.
• Compare what Ollama would cost you in electricity versus a cloud API using the Local vs Cloud panel.
• Check whether a bigger model fits before downloading it — select your GPU here and sort by fit.