Blog · Concepts · September 20, 2026 · 6 min read

GGUF Files Explained: What You’re Actually Downloading

When Ollama “pulls a model,” it downloads one or more files in the GGUF format — a single-file container that holds the model’s weights in a quantized form, plus metadata (architecture, tokenizer, context settings) that runners like llama.cpp need to execute it. One file, self-describing, portable between tools. That is the whole idea.

Reading the name

Take a typical filename: qwen2.5-32b-instruct-q4_k_m.gguf. Left to right:

• qwen2.5-32b-instruct — the base model: family, parameter count, and variant (instruct/chat-tuned vs base).
• q4_k_m — the quantization: ~4.5 bits per weight with the K-quant medium scheme. See our quantization guide for what each level trades away.

The same model exists in many quantization levels, and each level is a different file with a different size — roughly 0.5 GB per billion parameters at Q4, double at Q8, quadruple at FP16. That is why the same 32B model can be an 18 GB download or a 65 GB one.

Where the files live

• Ollama: Windows C:\Users\<you>\.ollama\models, macOS/Linux ~/.ollama/models — movable to another drive via the OLLAMA_MODELS environment variable.
• LM Studio: a models folder you choose in its settings, organized by creator and model.
• Direct downloads: any .gguf file from a model hub can be imported manually into both tools.

Split files and sharded models

Big models arrive as numbered parts — ...-00001-of-00004.gguf and so on. Keep all parts in the same folder; the runner stitches them automatically. Never delete “one part” to save space — the model becomes unusable; remove the whole set instead.

Disk hygiene

• List what you own with ollama list; sizes shown are the on-disk GGUF sizes.
• Remove unused variants with ollama rm — a forgotten FP16 duplicate can free 30+ GB instantly.
• Before downloading a new size, confirm it fits your hardware in the compatibility panel — disk space and VRAM have to agree.

That is GGUF in a nutshell: a portable, self-describing box of quantized weights. Decode the name and you know the quality, the size, and whether it fits — before spending the download.