Local AI

Can It Run? Local LLM GPU Compatibility Matrix (2026)

Can your GPU run that local LLM? A ✅/❌ compatibility matrix of 12 GPUs × 12 popular models (Llama, Qwen, Gemma, Mistral, DeepSeek) — VRAM fit at Q4, tokens/sec, and the cheapest GPU that runs each model.

Wondering whether your graphics card can run a given local LLM? This is the quick-answer matrix. Rows are GPUs (by VRAM), columns are popular open models (by size). A ✅ means the model fits in that card’s VRAM at Q4_K_M quantization — the recommended balance of quality and size. Pick your VRAM from the filter, or click any cell for the full breakdown: exact VRAM, expected tokens/sec, the right quant level, and (if it doesn’t fit) the cheapest GPU that does.

Your GPU VRAM: ✅ fits (Q4) · ❌ needs more VRAM

GPU ↓ / Model →	Llama 3.2 3B3B · 2.2GB	Qwen 2.5 7B7B · 4.5GB	Llama 3.1 8B8B · 5GB	Gemma 3 12B12B · 7.5GB	Phi-4 14B14B · 8.5GB	Qwen 2.5 14B14B · 8.5GB	Mistral Small 3 24B24B · 14GB	Gemma 2 27B27B · 16GB	Qwen 2.5 32B32B · 19GB	DeepSeek Coder V2 33B33B · 19.5GB	Llama 3.3 70B70B · 40GB	Qwen 2.5 72B72B · 41GB
RTX 4060 8GB8GB · ~$300	✅	✅	✅	✅	❌	❌	❌	❌	❌	❌	❌	❌
RTX 3080 10GB10GB · ~$400 used	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌	❌	❌
RTX 3060 12GB12GB · ~$300	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌	❌	❌
RTX 4070 12GB12GB · ~$550	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌	❌	❌
RTX 4060 Ti 16GB16GB · ~$450	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌
RTX 4070 Ti Super 16GB16GB · ~$800	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌
RTX 4080 Super 16GB16GB · ~$1,000	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌	❌	❌
RTX 3090 24GB24GB · ~$800 used	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
Radeon RX 7900 XTX 24GB24GB · ~$900	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
RTX 4090 24GB24GB · ~$1,800	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
RTX 5090 32GB32GB · ~$2,000+	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	❌	❌
Mac Studio M4 Max 128GB128GB · ~$3,500	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅	✅

✅ = fits in VRAM at Q4_K_M quantization (the recommended balance of quality and size). Click any cell for the full breakdown, expected tokens/sec, and the cheapest GPU that runs it. Estimates — verify against the hardware checker.

How to use this

Know your card? Find its row and scan across — green cells are models you can run today.
Shopping for a GPU? Find the model you want to run, then pick the cheapest card with a ✅ in that column.
Want the exact numbers? Use the LLM Hardware Checker (your GPU + RAM → every model you can run) and the VRAM requirements table.

New to running models locally? Start with the Local LLMs in 2026 guide.