MLX or llama.cpp on a Mac?
Ollama, LM Studio and most of what you will download are front ends over two engines: llama.cpp, which runs everywhere and reads GGUF files, and MLX, Apple’s own framework. Same model, same laptop, same 4-bit quantisation, measured with the order reversed so the fan could not pick the winner.
| engine | TTFT, 4K prompt | per token | decode | spread |
|---|---|---|---|---|
| MLX, 4-bit | 16.4 s | 33.3 ms | 30.1 tok/s | 2.0% |
| llama.cpp, Q4_K_M | 20.3 s | 58.7 ms | 17.0 tok/s | 4.1% |
MLX decodes 1.8 times faster and reaches first token 1.2 times sooner. Both differences are far outside the run-to-run spread, which is the only reason to trust them: on a fanless machine identical work ran in 16.6 seconds cold and 28.6 seconds after an hour, a 72% drift on the same prompt, so any comparison that was not run back to back and then reversed measures the temperature of the laptop. Context size was ruled out as a cause, 8K against 32K on llama.cpp moved memory and not speed. Some of the gap is engine and some is the quantisation formats not being identical, and this measurement cannot separate them. It is also one chip, one macOS version and one model, so treat it as the default rather than the law: MLX first on a Mac, and llama.cpp for anything MLX does not run.
Two defaults matter more than the engine. The first is llama.cpp’s prompt cache budget, --cache-ram, which defaults to 8 GiB, half of a 16 GB laptop. It keeps a full KV snapshot per distinct prompt, hundreds of MB each, and a workload with little prompt reuse fills it relentlessly. On the measured machine the server sat at 9,358 MB, swap went from 1.9 to 14.2 GB and free disk fell to 596 MB. Capped at 512 MiB the same server sat at 1,223 MB and swap fell during the run, with prefix reuse for an appended turn unaffected, because extending the active slot is slot-local and needs no prompt cache. One flag, eight times less memory, nothing lost. The second is reasoning: llama.cpp defaults a thinking model’s reasoning on and MLX defaults it off, so an unpinned comparison between them is a comparison of two different tasks.
The engines allocate memory differently, and the memory numbers you will read are wrong on a Mac. llama.cpp pre-allocates its whole KV cache at start, so a 32K context with automatic server slots was killed during model load, twice, before any inference. MLX grows on demand and never came close. And RSS, the figure every process monitor shows, read 1,826 MB for a loaded 3 GB model and fell after inference; Metal pages are accounted elsewhere. phys_footprint read 3,187 MB, which is true. llama.cpp also maps the GGUF file rather than loading it, so its footprint can be smaller than its own model file. Compare memory within one engine, never between two.
KV cache quantisation buys context, not speed. TurboQuant against a plain cache on the same prompt, in both orders: peak memory 8,235 MB against 7,568 MB at 16K, an 8% saving that grows with context and would be steeper on a model that caches every layer. A first run also showed it 1.24 times faster. With the arms reversed that vanished; the baseline arm had simply run hot. The measured spread on decode under sustained load reached 82%, so any speed claim smaller than that is noise in either direction, and that one was withdrawn.
What this measurement leaves out
Anything but one M2 Air, one 4B model, one macOS version and one build of each engine. Both move monthly. The order of the engines is likely to hold on Apple silicon generally, since MLX is the substrate the other Mac engines build on; the size of the gap is not.
An unexplained six seconds. llama.cpp’s own timing reported 14.2 seconds of prefill for a request the client saw take 20.2 seconds to first token; MLX’s self-report matched its client-side figure within a second. Both engines were measured the same way, so the comparison stands, but something costs llama.cpp time between finishing the prompt and delivering a token, and it was not found.
Quality. Two 4-bit formats were compared on speed and memory only. Nothing here says which one writes the better answer.