at-home/run-on/m3ultra-256
What can I run on a Mac Studio M3 Ultra with 256 GB?
Of its 256 GB, about 202 GB is left for a model's weights and cache, because macOS caps how much memory the GPU may pin and the engine needs some for itself. The largest model below that runs with room for a 8K conversation is Llama 3.3 70B at Q8_0, decoding at up to 9 tokens a second.
| Model | Best fit | Download | Longest context | Decode, up to |
|---|---|---|---|---|
| Llama 3.2 3B | Q8_0 | 3.4 GB | 1.7M tokens | 193 tok/s |
| Qwen3.5 4B | Q8_0 | 4.3 GB | 6.0M tokens | 154 tok/s |
| Llama 3.1 8B | Q8_0 | 8.5 GB | 1.5M tokens | 77 tok/s |
| Qwen3 8B | Q8_0 | 8.7 GB | 1.3M tokens | 75 tok/s |
| Qwen3.5 9B | Q8_0 | 9.6 GB | 5.9M tokens | 69 tok/s |
| Qwen3 14B | Q8_0 | 15.7 GB | 1.1M tokens | 42 tok/s |
| gpt-oss 20B | MXFP4, as shipped | 12.8 GB | 7.7M tokens | 51 tok/s |
| Qwen3 32B | Q8_0 | 34.9 GB | 622K tokens | 19 tok/s |
| Llama 3.3 70B | Q8_0 | 74.4 GB | 379K tokens | 9 tok/s |
Best fit is the highest-quality quantisation that still leaves room for a 8K context. It stops at Q8 rather than FP16, because Q8 is indistinguishable from FP16 in practice and reads half the bytes, so it decodes about twice as fast on a machine like this.
Decode is a ceiling: 80% of this machine’s 819 GB/s divided by the bytes a token reads, at the start of a conversation. On a measured 16 GB MacBook Air that arithmetic predicted 35.6 tokens a second and MLX delivered 31.4; llama.cpp delivered 17 on the same model. Expect your engine to land at or below it.
Longest context assumes an unquantised KV cache. Quantising the cache to Q8 roughly doubles it. Reading a long prompt also needs working memory the table does not count, so leave a margin.