LLMs you can run locally
Open-weight models small enough to run on a single consumer GPU, a workstation card or a high-memory Mac, with the memory each one needs.
Running a model on your own machine keeps data private, works offline and costs nothing per token once you own the hardware. The limit is memory: the weights must fit in GPU memory (or Apple unified memory), with room left for the KV cache.
Memory figures below are for the weights at 4-bit quantization (about 4.85 bits per weight), the most common setting for local use. Higher precision roughly doubles (8-bit) or quadruples (16-bit) the requirement.
The shortlist
Fits a single consumer GPU or a 32 GB+ Mac (about 36B parameters or fewer)
| Model | Tier | Weights | Context | Weights at 4-bit | Input / 1M | Output / 1M | Inputs |
|---|---|---|---|---|---|---|---|
| Ministral 3 3BMistral AI | Compact | Open weights | 256K | 2 GB | $0.10 | $0.10 | Text, Images |
| Phi-4-mini-instructMicrosoft (Phi) | Compact | Open weights | 128K | 2 GB | — | — | Text |
| Gemma 4 E4BGoogle | Compact | Open weights | 128K | 5 GB | — | — | Text, Images, Audio, Video |
| Ministral 3 8BMistral AI | Compact | Open weights | 256K | 5 GB | $0.15 | $0.15 | Text, Images |
| Granite 4.2 8BIBM | Compact | Open weights | 128K | 5 GB | $0.06 | $0.25 | Text |
| Ministral 3 14BMistral AI | Efficient | Open weights | 256K | 8 GB | $0.20 | $0.20 | Text, Images |
| Phi-4-reasoning-vision-15BMicrosoft (Phi) | Compact | Open weights | 16K | 9 GB | — | — | Text, Images |
| gpt-oss-20bOpenAI | Compact | Open weights | 128K | 13 GB | $0.075 | $0.30 | Text |
| Gemma 4 26B A4BGoogle | Compact | Open weights | 256K | 16 GB | $0.07 | $0.34 | Text, Images, Video |
| Qwen3.8-27BAlibaba (Qwen) | Efficient | Open weights | 256K | 17 GB | $0.80 | $4 | Text, Images, Video |
| Granite 4.2 30BIBM | Efficient | Open weights | 128K | 18 GB | $0.16 | $0.65 | Text |
| Muse Glimmer 30BMeta | Efficient | Open weights | 128K | 18 GB | $0.35 | $1.50 | Text, Images, Video |
| NVIDIA Nemotron 3.5 Lightning 30B-A3BNVIDIA | Compact | Open weights | 1M | 18 GB | $0.05 | $0.20 | Text |
| Gemma 4 31BGoogle | Efficient | Open weights | 256K | 19 GB | $0.20 | $0.40 | Text, Images, Video |
| Olmo 3.1 32B ThinkAi2 (Allen Institute for AI) | Efficient | Open weights | 64K | 19 GB | — | — | Text |
| Qwen3.6-35B-A3BAlibaba (Qwen) | Compact | Open weights | 256K | 22 GB | $0.10 | $0.95 | Text, Images, Video |
Needs a workstation GPU, two GPUs or a 64-128 GB Mac
| Model | Tier | Weights | Context | Weights at 4-bit | Input / 1M | Output / 1M | Inputs |
|---|---|---|---|---|---|---|---|
| gpt-oss-120bOpenAI | Advanced | Open weights | 128K | 71 GB | $0.15 | $0.60 | Text |
| Mistral Small 4Mistral AI | Efficient | Open weights | 256K | 72 GB | $0.15 | $0.60 | Text, Images |
| Mistral Medium 3.5Mistral AI | Advanced | Open weights | 256K | 77 GB | $1.50 | $7.50 | Text, Images |
Ordered by size. Tiers are our editorial grouping of provider positioning and public benchmark standings, not our own test results. How we rank.
How to choose
- 8-12 GB GPUs: models up to about 8B parameters at 4-bit.
- 16-24 GB GPUs: up to roughly 30B at 4-bit with a moderate context.
- Macs with 64-128 GB of unified memory: up to about 120B mixture-of-experts models at 4-bit.
- Size a specific setup with the VRAM calculator and follow our guide to running LLMs locally.
Frequently asked questions
What is the best LLM to run locally?
The best local model is the most capable one that fits your memory with room for context. On a 24 GB GPU that's usually a model of around 20-30B parameters at 4-bit; on a 128 GB Mac, mixture-of-experts models above 100B become practical.
How much RAM do I need to run an LLM?
For GPU inference, count GPU memory: about 0.6 GB per billion parameters at 4-bit for the weights, plus the KV cache. On Apple Silicon, the GPU can use most of the unified memory, so a 32 GB Mac comfortably runs models in the 8-14B range.
Other shortlists
Planning to self-host?
Get a second opinion on hardware, serving stack and break-even before you buy or rent GPUs.
- Free and vendor-neutral: no model provider pays for placement.
- Your details go only to partners you agree to.