RunAIHow.comLearn How To Run LocallyUpdated: OCTOBER 2026
Hardware
Fields auto-fill from the selected card. Edit any value to switch to a custom GPU.
- Total VRAM
- 24 GB
- Bandwidth
- 1008 GB/s
- Total TDP
- 450 W
VRAM and TDP scale with GPU count; bandwidth is per-GPU.
Model Compatibility
Context adds KV-cache memory: kvGBPer1kTokens × (tokens / 1000), per model.
| Model | VRAM needed | Fit | Speed est. |
|---|---|---|---|
Kimi K3 Moonshot AI · 2.8T (104B active) | 5216 GB 5215 GB + 0.42 GB KV | Won't Fit | 0.19 tok/s |
Qwen3.8 2.4T Alibaba · 2.4T (95B active) | 4471 GB 4470 GB + 0.75 GB KV | Won't Fit | 0.23 tok/s |
DeepSeek V4 Pro DeepSeek · 1.6T | 2981 GB 2980 GB + 0.28 GB KV | Won't Fit | 0.34 tok/s |
MiMo V2.5 Pro Xiaomi · 1.0T | 1900 GB 1900 GB + 0.35 GB KV | Won't Fit | 0.53 tok/s |
Kimi K2.7 Code Moonshot AI · 1T (32B active) | 1864 GB 1863 GB + 1.07 GB KV | Won't Fit | 0.54 tok/s |
Kimi K2.6 Moonshot AI · 1T (32B active) | 1864 GB 1863 GB + 1.07 GB KV | Won't Fit | 0.54 tok/s |
Kimi K2.5 Moonshot AI · 1T (32B active) | 1864 GB 1863 GB + 1.07 GB KV | Won't Fit | 0.54 tok/s |
MiMo V2 Pro Xiaomi · 1T (43B active) | 1863 GB 1863 GB + 0.35 GB KV | Won't Fit | 0.54 tok/s |
Kimi K2 Moonshot AI · 1T (32B active) | 1864 GB 1863 GB + 1.07 GB KV | Won't Fit | 0.54 tok/s |
GLM-5.3 Z.ai · 744B (40B active) | 1386 GB 1386 GB + 0.35 GB KV | Won't Fit | 0.73 tok/s |
GLM-5.2 Z.ai · 744B (40B active) | 1386 GB 1386 GB + 0.35 GB KV | Won't Fit | 0.73 tok/s |
GLM-5.1 Z.ai · 744B (40B active) | 1386 GB 1386 GB + 0.35 GB KV | Won't Fit | 0.73 tok/s |
GLM-5 Z.ai · 744B (40B active) | 1386 GB 1386 GB + 0.35 GB KV | Won't Fit | 0.73 tok/s |
DeepSeek V3.2 DeepSeek · 685B | 1277 GB 1276 GB + 1.07 GB KV | Won't Fit | 0.79 tok/s |
DeepSeek R1 DeepSeek · 671B | 1251 GB 1250 GB + 1.07 GB KV | Won't Fit | 0.81 tok/s |
DeepSeek V4.1 Flash DeepSeek · 552B (16B active) | 1029 GB 1028 GB + 0.42 GB KV | Won't Fit | 0.98 tok/s |
MiniMax Text 01 MiniMax · 456B (45.9B active) | 850 GB 849 GB + 0.31 GB KV | Won't Fit | 1.19 tok/s |
MiniMax M1 MiniMax · 456B (45.9B active) | 850 GB 849 GB + 0.31 GB KV | Won't Fit | 1.19 tok/s |
MiniMax M3 MiniMax · 428B (23B active) | 797 GB 797 GB + 0.25 GB KV | Won't Fit | 1.26 tok/s |
Qwen3.5 397B Alibaba · 397B (17B active) | 740 GB 739 GB + 0.50 GB KV | Won't Fit | 1.36 tok/s |
GLM-4.6 Z.ai · 357B (32B active) | 667 GB 665 GB + 1.55 GB KV | Won't Fit | 1.52 tok/s |
GLM-4.7 Z.ai · 355B (32B active) | 663 GB 661 GB + 1.55 GB KV | Won't Fit | 1.52 tok/s |
GLM-4.5 Z.ai · 355B (32B active) | 663 GB 661 GB + 1.55 GB KV | Won't Fit | 1.52 tok/s |
GLM-5.3 Flash Z.ai · 321B | 598 GB 598 GB + 0.19 GB KV | Won't Fit | 1.69 tok/s |
MiMo V2.5 Xiaomi · 310B | 578 GB 577 GB + 0.28 GB KV | Won't Fit | 1.75 tok/s |
Qwen3 235B A22B Alibaba · 235B (22B active) | 439 GB 438 GB + 1.47 GB KV | Won't Fit | 2.30 tok/s |
MiniMax M2 MiniMax · 230B (10B active) | 429 GB 428 GB + 0.48 GB KV | Won't Fit | 2.35 tok/s |
Step 3.7 Flash StepFun · 198B (11B active) | 370 GB 369 GB + 0.98 GB KV | Won't Fit | 2.73 tok/s |
Step 3.5 Flash StepFun · 196B (11B active) | 366 GB 365 GB + 0.98 GB KV | Won't Fit | 2.76 tok/s |
Qwen3.8 Flash Next Alibaba · 180B | 336 GB 335 GB + 0.38 GB KV | Won't Fit | 3.01 tok/s |
Ling 3.0 Flash VL InclusionAI · 124B (5.5B active) | 232 GB 231 GB + 0.75 GB KV | Won't Fit | 4.36 tok/s |
Mistral Small 4 Mistral AI · 119B (6B active) | 223 GB 222 GB + 1.50 GB KV | Won't Fit | 4.55 tok/s |
GPT-OSS 120B OpenAI · 116.8B (5.1B active) | 218 GB 218 GB + 0.56 GB KV | Won't Fit | 4.63 tok/s |
GLM-4.5 Air Z.ai · 106B (12B active) | 198 GB 197 GB + 0.81 GB KV | Won't Fit | 5.11 tok/s |
Qwen2.5 72B Alibaba · 72.7B | 138 GB 135 GB + 2.50 GB KV | Won't Fit | 7.44 tok/s |
Llama 3.3 70B Meta · 70.5B | 134 GB 131 GB + 2.50 GB KV | Won't Fit | 7.67 tok/s |
Mixtral 8x7B Mistral AI · 46.7B (12.9B active) | 88.0 GB 87.0 GB + 1.00 GB KV | Won't Fit | 11.6 tok/s |
Qwen3.6 35B A3B Alibaba · 35B (3B active) | 65.6 GB 65.2 GB + 0.38 GB KV | Won't Fit | 15.5 tok/s |
MiniMax H3 MiniMax · 33B | 62.6 GB 61.5 GB + 1.12 GB KV | Won't Fit | 16.4 tok/s |
Qwen3 32B Alibaba · 32.8B | 63.1 GB 61.1 GB + 2.00 GB KV | Won't Fit | 16.5 tok/s |
Qwen2.5 32B Alibaba · 32.8B | 63.0 GB 61.0 GB + 2.00 GB KV | Won't Fit | 16.5 tok/s |
Qwen2.5 Coder 32B Alibaba · 32.8B | 63.0 GB 61.0 GB + 2.00 GB KV | Won't Fit | 16.5 tok/s |
DeepSeek-R1 Distill Qwen 32B DeepSeek · 32.8B | 63.0 GB 61.0 GB + 2.00 GB KV | Won't Fit | 16.5 tok/s |
Gemma 4 31B Google · 31B | 61.6 GB 57.7 GB + 3.87 GB KV | Won't Fit | 17.5 tok/s |
Qwen3 30B A3B Instruct 2507 Alibaba · 30.5B (3.3B active) | 57.6 GB 56.8 GB + 0.75 GB KV | Won't Fit | 17.7 tok/s |
Qwen3.8 27B Alibaba · 27.9B | 52.5 GB 52.0 GB + 0.50 GB KV | Won't Fit | 19.4 tok/s |
Gemma 3 27B Google · 27.4B | 54.9 GB 51.0 GB + 3.87 GB KV | Won't Fit | 19.8 tok/s |
Gemma 2 27B Google · 27.2B | 53.6 GB 50.7 GB + 2.88 GB KV | Won't Fit | 19.9 tok/s |
Gemma 4 26B A4B Google · 26B (3.8B active) | 51.4 GB 48.4 GB + 3.00 GB KV | Won't Fit | 20.8 tok/s |
GPT-OSS 20B OpenAI · 20.9B (3.6B active) | 39.3 GB 38.9 GB + 0.38 GB KV | Won't Fit | 25.9 tok/s |
Qwen-Image 20B Alibaba · 20B | 37.3 GB 37.3 GB + 0.00 GB KV | Won't Fit | 27.1 tok/s |
HiDream-I1 17B HiDream.ai · 17B | 31.7 GB 31.7 GB + 0.00 GB KV | Won't Fit | 31.8 tok/s |
Qwen3 14B Alibaba · 14.8B | 28.8 GB 27.6 GB + 1.25 GB KV | Won't Fit | 36.6 tok/s |
Qwen2.5 14B Alibaba · 14.8B | 29.0 GB 27.5 GB + 1.50 GB KV | Won't Fit | 36.6 tok/s |
Phi-4 14B Microsoft · 14.7B | 33.6 GB 27.3 GB + 6.25 GB KV | Won't Fit | 36.9 tok/s |
Wan 2.2 14B Alibaba · 14B | 26.1 GB 26.1 GB + 0.00 GB KV | Won't Fit | 38.7 tok/s |
HunyuanVideo 13B Tencent · 13B | 24.2 GB 24.2 GB + 0.00 GB KV | Won't Fit | 41.6 tok/s |
Gemma 3 12B Google · 11.8B | 25.0 GB 22.0 GB + 3.00 GB KV | Won't Fit | 45.9 tok/s |
Mochi 1 10B Genmo · 10B | 18.6 GB 18.6 GB + 0.00 GB KV | Excellent Fit | 54.1 tok/s |
Gemma 2 9B Google · 9.2B | 19.8 GB 17.2 GB + 2.62 GB KV | Excellent Fit | 58.6 tok/s |
Qwen3 8B Alibaba · 8.2B | 16.4 GB 15.3 GB + 1.12 GB KV | Excellent Fit | 66.1 tok/s |
Llama 3.1 8B Meta · 8.0B | 16.0 GB 15.0 GB + 1.00 GB KV | Excellent Fit | 67.4 tok/s |
DeepSeek-R1 Distill Llama 8B DeepSeek · 8.0B | 16.0 GB 15.0 GB + 1.00 GB KV | Excellent Fit | 67.4 tok/s |
Qwen2.5 7B Alibaba · 7.6B | 14.6 GB 14.2 GB + 0.44 GB KV | Excellent Fit | 71.0 tok/s |
DeepSeek-R1 Distill Qwen 7B DeepSeek · 7.6B | 14.6 GB 14.2 GB + 0.44 GB KV | Excellent Fit | 71.0 tok/s |
Mistral 7B v0.3 Mistral AI · 7.3B | 17.5 GB 13.5 GB + 4.00 GB KV | Excellent Fit | 74.6 tok/s |
MiMo 7B Xiaomi · 7B | 14.0 GB 13.0 GB + 1.00 GB KV | Excellent Fit | 77.3 tok/s |
Gemma 3 4B Google · 4.3B | 9.07 GB 8.01 GB + 1.06 GB KV | Excellent Fit | 126 tok/s |
Qwen3 4B Alibaba · 4B | 8.58 GB 7.45 GB + 1.12 GB KV | Excellent Fit | 135 tok/s |
Phi-3.5 Mini 3.8B Microsoft · 3.8B | 10.1 GB 7.12 GB + 3.00 GB KV | Excellent Fit | 142 tok/s |
Stable Diffusion XL Stability AI · 3.5B | 6.52 GB 6.52 GB + 0.00 GB KV | Excellent Fit | 155 tok/s |
Llama 3.2 3B Meta · 3.2B | 6.85 GB 5.98 GB + 0.87 GB KV | Excellent Fit | 169 tok/s |
LTX-Video 2B Lightricks · 2B | 3.73 GB 3.73 GB + 0.00 GB KV | Excellent Fit | 271 tok/s |
MusicGen Medium 1.5B Meta · 1.5B | 2.79 GB 2.79 GB + 0.00 GB KV | Excellent Fit | 361 tok/s |
Llama 3.2 1B Meta · 1.2B | 2.56 GB 2.31 GB + 0.25 GB KV | Excellent Fit | 436 tok/s |
Stable Audio Open Stability AI · 1.2B | 2.24 GB 2.24 GB + 0.00 GB KV | Excellent Fit | 451 tok/s |
Gemma 3 1B Google · 1.0B | 2.75 GB 1.94 GB + 0.81 GB KV | Excellent Fit | 520 tok/s |
F5-TTS 336M Open source · 0.3B | 0.63 GB 0.63 GB + 0.00 GB KV | Excellent Fit | 1611 tok/s |
Kokoro TTS 82M Open source · 0.1B | 0.15 GB 0.15 GB + 0.00 GB KV | Excellent Fit | 6600 tok/s |
GPT-6 Astra OpenAI · — | Cloud only or closed source | — | — |
GPT-5.6 OpenAI · — | Cloud only or closed source | — | — |
GPT-5.5 OpenAI · — | Cloud only or closed source | — | — |
GPT-5.4 OpenAI · — | Cloud only or closed source | — | — |
GPT-5.3 Codex OpenAI · — | Cloud only or closed source | — | — |
Gemini 3 Pro Google DeepMind · — | Cloud only or closed source | — | — |
Claude Opus 5 Anthropic · — | Cloud only or closed source | — | — |
Claude Mythos 5.1 Anthropic · — | Cloud only or closed source | — | — |
Claude Fable 5 Anthropic · — | Cloud only or closed source | — | — |
Claude Fable 5.1 Anthropic · — | Cloud only or closed source | — | — |
Claude Opus 4.8 Anthropic · — | Cloud only or closed source | — | — |
Claude Opus 4.7 Anthropic · — | Cloud only or closed source | — | — |
Claude Opus 4.6 Anthropic · — | Cloud only or closed source | — | — |
Claude Sonnet 4.6 Anthropic · — | Cloud only or closed source | — | — |
Grok 4.5 xAI · — | Cloud only or closed source | — | — |
Grok 4 xAI · — | Cloud only or closed source | — | — |
Qwen3.8 Max Alibaba · — | Cloud only or closed source | — | — |
Gemini 3.8 Flash Google DeepMind · — | Cloud only or closed source | — | — |
Gemini 3.7 Flash Google DeepMind · — | Cloud only or closed source | — | — |
Gemini 3.6 Flash Google DeepMind · — | Cloud only or closed source | — | — |
Gemini 3.5 Flash Google DeepMind · — | Cloud only or closed source | — | — |
Gemini 3.5 Flash Lite Google DeepMind · — | Cloud only or closed source | — | — |
GPT-5.6 Luna OpenAI · — | Cloud only or closed source | — | — |
Claude Haiku 4.5 Anthropic · — | Cloud only or closed source | — | — |
Mercury 2.5 Inception · — | Cloud only or closed source | — | — |
Token Cost
≈ 21 tokens · 83 chars (4 chars ≈ 1 token)
Pre-defined estimates: typical prompt and response tokens per request plus a suggested daily volume. Hosted image, video, and music services usually bill per image, second, or track instead of per token.
Estimated monthly volume: ≈ 6K in / 154K out tokens (300 requests)
| Model | $ / 1M in | $ / 1M out | Est. per request | Est. per month |
|---|---|---|---|---|
F5-TTS 336M Open source | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
HiDream-I1 17B HiDream.ai | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
HunyuanVideo 13B Tencent | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Kokoro TTS 82M Open source | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
LTX-Video 2B Lightricks | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Mochi 1 10B Genmo | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
MusicGen Medium 1.5B Meta | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Qwen-Image 20B Alibaba | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Stable Audio Open Stability AI | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Stable Diffusion XL Stability AI | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Wan 2.2 14B Alibaba | $0.0000 | $0.0000 | $0.0000 | $0.0000 |
Llama 3.2 1B Meta | $0.02 | $0.03 | $0.0000 | $0.0047 |
Gemma 3 1B Google | $0.03 | $0.05 | $0.0000 | $0.0079 |
Llama 3.2 3B Meta | $0.06 | $0.06 | $0.0000 | $0.0096 |
Mistral 7B v0.3 Mistral AI | $0.07 | $0.07 | $0.0000 | $0.01 |
Gemma 3 4B Google | $0.05 | $0.10 | $0.0001 | $0.02 |
Qwen3 4B Alibaba | $0.05 | $0.10 | $0.0001 | $0.02 |
Phi-3.5 Mini 3.8B Microsoft | $0.10 | $0.10 | $0.0001 | $0.02 |
Qwen2.5 7B Alibaba | $0.10 | $0.10 | $0.0001 | $0.02 |
Qwen3 8B Alibaba | $0.10 | $0.10 | $0.0001 | $0.02 |
Mercury 2.5 Inception | $0.04 | $0.15 | $0.0001 | $0.02 |
Ling 3.0 Flash VL InclusionAI | $0.06 | $0.18 | $0.0001 | $0.03 |
MiMo 7B Xiaomi | $0.05 | $0.20 | $0.0001 | $0.03 |
Gemma 3 12B Google | $0.10 | $0.20 | $0.0001 | $0.03 |
Gemma 2 9B Google | $0.20 | $0.20 | $0.0001 | $0.03 |
Qwen2.5 14B Alibaba | $0.25 | $0.25 | $0.0001 | $0.04 |
Llama 3.1 8B Meta | $0.10 | $0.30 | $0.0002 | $0.05 |
Mistral Small 4 Mistral AI | $0.10 | $0.30 | $0.0002 | $0.05 |
Gemma 3 27B Google | $0.20 | $0.30 | $0.0002 | $0.05 |
Gemma 2 27B Google | $0.30 | $0.30 | $0.0002 | $0.05 |
Gemma 4 26B A4B Google | $0.20 | $0.35 | $0.0002 | $0.06 |
Gemini 3.5 Flash Lite Google DeepMind | $0.10 | $0.40 | $0.0002 | $0.06 |
MiniMax H3 MiniMax | $0.10 | $0.40 | $0.0002 | $0.06 |
Qwen3.8 Flash Next Alibaba | $0.10 | $0.40 | $0.0002 | $0.06 |
Gemma 4 31B Google | $0.25 | $0.40 | $0.0002 | $0.06 |
Qwen2.5 32B Alibaba | $0.35 | $0.40 | $0.0002 | $0.06 |
Qwen2.5 Coder 32B Alibaba | $0.35 | $0.40 | $0.0002 | $0.06 |
Qwen2.5 72B Alibaba | $0.40 | $0.40 | $0.0002 | $0.06 |
DeepSeek V3.2 DeepSeek | $0.28 | $0.42 | $0.0002 | $0.07 |
Phi-4 14B Microsoft | $0.12 | $0.48 | $0.0002 | $0.07 |
GPT-OSS 20B OpenAI | $0.10 | $0.50 | $0.0003 | $0.08 |
DeepSeek-R1 Distill Llama 8B DeepSeek | $0.15 | $0.55 | $0.0003 | $0.09 |
DeepSeek-R1 Distill Qwen 7B DeepSeek | $0.15 | $0.55 | $0.0003 | $0.09 |
DeepSeek V4.1 Flash DeepSeek | $0.15 | $0.60 | $0.0003 | $0.09 |
GLM-5.3 Flash Z.ai | $0.15 | $0.60 | $0.0003 | $0.09 |
Qwen3 14B Alibaba | $0.15 | $0.60 | $0.0003 | $0.09 |
Qwen3 30B A3B Instruct 2507 Alibaba | $0.15 | $0.60 | $0.0003 | $0.09 |
Qwen3 32B Alibaba | $0.15 | $0.60 | $0.0003 | $0.09 |
Step 3.5 Flash StepFun | $0.15 | $0.60 | $0.0003 | $0.09 |
Step 3.7 Flash StepFun | $0.15 | $0.60 | $0.0003 | $0.09 |
GLM-4.5 Air Z.ai | $0.20 | $0.60 | $0.0003 | $0.09 |
Qwen3.6 35B A3B Alibaba | $0.20 | $0.60 | $0.0003 | $0.09 |
Qwen3.8 27B Alibaba | $0.20 | $0.60 | $0.0003 | $0.09 |
Mixtral 8x7B Mistral AI | $0.60 | $0.60 | $0.0003 | $0.10 |
Qwen3 235B A22B Alibaba | $0.20 | $0.70 | $0.0004 | $0.11 |
GPT-OSS 120B OpenAI | $0.15 | $0.75 | $0.0004 | $0.12 |
Llama 3.3 70B Meta | $0.60 | $0.80 | $0.0004 | $0.13 |
GPT-5.6 Luna OpenAI | $0.20 | $1.20 | $0.0006 | $0.19 |
MiniMax Text 01 MiniMax | $0.20 | $1.20 | $0.0006 | $0.19 |
MiMo V2.5 Xiaomi | $0.30 | $1.20 | $0.0006 | $0.19 |
MiniMax M2 MiniMax | $0.30 | $1.20 | $0.0006 | $0.19 |
MiniMax M3 MiniMax | $0.30 | $1.20 | $0.0006 | $0.19 |
DeepSeek-R1 Distill Qwen 32B DeepSeek | $0.40 | $1.20 | $0.0006 | $0.19 |
DeepSeek V4 Pro DeepSeek | $0.58 | $1.74 | $0.0009 | $0.27 |
MiniMax M1 MiniMax | $0.40 | $2.10 | $0.0011 | $0.33 |
DeepSeek R1 DeepSeek | $0.55 | $2.19 | $0.0011 | $0.34 |
GLM-4.5 Z.ai | $0.55 | $2.20 | $0.0011 | $0.34 |
GLM-4.6 Z.ai | $0.60 | $2.20 | $0.0011 | $0.34 |
GLM-4.7 Z.ai | $0.60 | $2.20 | $0.0011 | $0.34 |
MiMo V2 Pro Xiaomi | $0.60 | $2.40 | $0.0012 | $0.37 |
MiMo V2.5 Pro Xiaomi | $0.60 | $2.40 | $0.0012 | $0.37 |
Qwen3.5 397B Alibaba | $0.60 | $2.40 | $0.0012 | $0.37 |
Qwen3.8 2.4T Alibaba | $0.60 | $2.40 | $0.0012 | $0.37 |
Gemini 3.5 Flash Google DeepMind | $0.30 | $2.50 | $0.0013 | $0.39 |
Gemini 3.6 Flash Google DeepMind | $0.30 | $2.50 | $0.0013 | $0.39 |
Gemini 3.7 Flash Google DeepMind | $0.30 | $2.50 | $0.0013 | $0.39 |
Gemini 3.8 Flash Google DeepMind | $0.30 | $2.50 | $0.0013 | $0.39 |
Kimi K2 Moonshot AI | $0.60 | $2.50 | $0.0013 | $0.39 |
Kimi K2.5 Moonshot AI | $0.60 | $2.50 | $0.0013 | $0.39 |
Kimi K2.6 Moonshot AI | $0.60 | $2.50 | $0.0013 | $0.39 |
GLM-5 Z.ai | $0.70 | $2.80 | $0.0014 | $0.43 |
GLM-5.1 Z.ai | $0.70 | $2.80 | $0.0014 | $0.43 |
GLM-5.2 Z.ai | $0.70 | $2.80 | $0.0014 | $0.43 |
GLM-5.3 Z.ai | $0.70 | $2.80 | $0.0014 | $0.43 |
Kimi K2.7 Code Moonshot AI | $0.60 | $3.00 | $0.0015 | $0.46 |
Kimi K3 Moonshot AI | $0.60 | $3.00 | $0.0015 | $0.46 |
Claude Haiku 4.5 Anthropic | $1.00 | $5.00 | $0.0026 | $0.77 |
Qwen3.8 Max Alibaba | $1.20 | $6.00 | $0.0031 | $0.93 |
GPT-5.3 Codex OpenAI | $1.25 | $10.00 | $0.0051 | $1.54 |
GPT-5.4 OpenAI | $1.25 | $10.00 | $0.0051 | $1.54 |
GPT-5.5 OpenAI | $1.25 | $10.00 | $0.0051 | $1.54 |
Gemini 3 Pro Google DeepMind | $2.00 | $12.00 | $0.0062 | $1.86 |
GPT-5.6 OpenAI | $2.00 | $12.00 | $0.0062 | $1.86 |
Claude Sonnet 4.6 Anthropic | $3.00 | $15.00 | $0.0077 | $2.32 |
Grok 4 xAI | $3.00 | $15.00 | $0.0077 | $2.32 |
Grok 4.5 xAI | $3.00 | $15.00 | $0.0077 | $2.32 |
Claude Fable 5 Anthropic | $4.00 | $20.00 | $0.01 | $3.10 |
Claude Fable 5.1 Anthropic | $4.00 | $20.00 | $0.01 | $3.10 |
Claude Mythos 5.1 Anthropic | $5.00 | $25.00 | $0.01 | $3.87 |
Claude Opus 4.6 Anthropic | $5.00 | $25.00 | $0.01 | $3.87 |
Claude Opus 4.7 Anthropic | $5.00 | $25.00 | $0.01 | $3.87 |
Claude Opus 4.8 Anthropic | $5.00 | $25.00 | $0.01 | $3.87 |
Claude Opus 5 Anthropic | $5.00 | $25.00 | $0.01 | $3.87 |
GPT-6 Astra OpenAI | $10.00 | $50.00 | $0.03 | $7.74 |
Local vs Cloud Crossover
Cumulative cost over 10 years. Hardware amortized over 36 months (the local line flattens after that); cloud scales with token volume. Break-even can also occur after amortization ends.
Common Questions and Concerns About Running AI Locally — FAQ
Quick answers to the questions we hear most. Open a question to read more.
Do I need an expensive GPU to run AI models locally?
No. Any modern PC can run small models on the CPU — just slower. A GPU makes everything dramatically faster, and 8 GB of VRAM is a comfortable entry point for popular 7–8B models. Pick your hardware in the Hardware panel to see exactly what your machine can run.
Does running AI locally send my data to the internet?
No. Once a model is downloaded, inference happens entirely on your machine — your prompts and documents never leave it. Tools like Ollama only listen on your own computer (localhost), and you can even pull models once and work fully offline. This is the main reason people choose local AI.
How much does it cost to run models locally every month?
Only electricity — typically $5–15/month depending on your hardware’s power draw and usage hours, plus the one-time hardware cost spread over its life. Enter your setup in the crossover panel to compare that against what the same usage would cost in cloud API fees.
Why does my model run out of memory even though “it should fit”?
Two usual suspects: context length (the KV-cache grows with every token and can rival the model itself — check the context slider in our compatibility panel) and headroom (your operating system and display already use 1–2 GB). Plan to keep at least 15% of VRAM free, which is what our “Excellent Fit” badge requires.
Are local models as good as ChatGPT and other cloud assistants?
For everyday tasks — writing, summarizing, coding help, Q&A over documents — modern open models at sensible quantization are genuinely competitive and fully private. The largest frontier cloud models still lead on the hardest reasoning, and they carry enormous knowledge. The practical approach: run locally for privacy, speed, and cost; reach for the cloud for the rare tasks that truly need frontier capability.
From the Blog
Welcome to RunAIHow: What This Site Is and How to Use It
New here? A two-minute tour of the three panels — compatibility, token cost, and local vs cloud — and how to answer 'can my PC run this AI model?' in under a minute.
September 20, 2026 · 5 min read · Read guide →
Getting Started with Ollama: Install and Run AI Models on Your Own Machine
A beginner-friendly guide to installing Ollama on Windows, macOS, and Linux, pulling your first model, using the built-in API, and picking a model that fits your GPU.
September 17, 2026 · 7 min read · Read guide →
Build Your Own Private ChatGPT: Ollama + Open WebUI on Any PC
Turn Ollama into a full ChatGPT-style experience with chat history, document uploads, and multi-user access — running entirely on your own hardware.
September 18, 2026 · 7 min read · Read guide →
How Much VRAM Do You Actually Need? Context, Quantization, and the Math Nobody Explains
Weights vs KV-cache, the real VRAM formula, and worked examples from 1B to 70B — how to size a GPU for the model and context length you actually want.
September 18, 2026 · 8 min read · Read guide →
Best Budget GPUs for Local AI in 2026: RTX 5060 Ti 16GB vs RTX 3090 vs RX 7900 XTX
VRAM capacity, memory bandwidth, power draw, and software support — which sub-$1000 card runs the models you care about fastest, and when used wins.
September 18, 2026 · 9 min read · Read guide →
Q4 vs Q8 vs FP16: What Quantization Actually Costs You in Quality and Memory
What those Q4_K_M labels really mean, how much quality each level gives up, why the sweet spot exists, and how to read our VRAM estimates correctly.
September 19, 2026 · 8 min read · Read guide →
Cloud API Bills Adding Up? Finding Your Local-vs-Cloud Break-Even Point
A worked financial example: GPU price, electricity, and token volume — how to tell after how many months your own hardware beats paying per token.
September 19, 2026 · 7 min read · Read guide →
LM Studio vs Ollama: Which Local AI App Should You Use?
Two popular ways to run models on your desktop — a practical comparison of setup, model discovery, interfaces, APIs, and who each tool is really for.
September 20, 2026 · 7 min read · Read guide →
Context Windows Explained: What 8K vs 128K Actually Means for You
What a context window holds, why models forget, how much of your documents actually fit, and what long contexts cost in VRAM.
September 20, 2026 · 7 min read · Read guide →
Why a '235B' Model Runs Like a 22B: Mixture-of-Experts Explained
Total parameters vs active parameters — how MoE models deliver big-model quality with small-model speed, and how to read names like Qwen3-235B-A22B.
September 20, 2026 · 6 min read · Read guide →
Local AI on Apple Silicon: How Far Can 16GB, 36GB, or 128GB Go?
Unified memory changes the rules — which Macs run which models, what the bandwidth numbers really deliver, and the honest limits of laptop-class chips.
September 20, 2026 · 7 min read · Read guide →
Tokens Per Second Explained: Why Memory Bandwidth Beats Everything
The decode-speed formula, what 10 vs 50 vs 100 tokens per second actually feels like, and why a slower 'gaming' GPU can beat a faster one at AI.
September 20, 2026 · 6 min read · Read guide →
PSUs, Cooling, and 24/7 Inference: Building a PC That Can Run AI All Day
Sustained AI loads are unlike gaming — power supply sizing, airflow, dust, and noise for a machine that pins its GPU for hours at a time.
September 20, 2026 · 7 min read · Read guide →
RAG vs Fine-Tuning: Making a Model Know Your Documents
Two very different ways to customize an AI — what each costs, when retrieval beats training, and how Open WebUI's document chat fits in.
September 20, 2026 · 7 min read · Read guide →
GGUF Files Explained: What You're Actually Downloading
Inside the file format behind Ollama and llama.cpp — how to read names like 'qwen2.5-32b-q4_k_m', where files live, and how to manage disk space.
September 20, 2026 · 6 min read · Read guide →
Buying a Used GPU for AI: The 10-Point Checklist
Local AI lives at 100% load for hours — how to test VRAM, thermals, and honesty before you hand over cash for a secondhand card.
September 20, 2026 · 8 min read · Read guide →
Prompt Caching and Batch APIs: Legit Ways to Cut Your Cloud Bill 50-90%
Before buying hardware, make sure you're not overpaying the cloud — cached inputs, batch processing, and model-tier choices that change the math.
September 20, 2026 · 6 min read · Read guide →
Building a Dual-GPU 48GB Rig for 70B Models: A Practical Guide
Two mid-range cards can out-muscle one flagship — pairing, layer splitting, motherboard and PSU planning, and the honest performance expectations.
September 20, 2026 · 8 min read · Read guide →
- More guides coming soon — GPU buying for local AI, quantization explained, and KV-cache deep dives. Visit the blog →