Will this model fit on my machine?

The weights are the fixed cost. A model’s parameter count times the bits per weight is the download, and it is the part everyone checks. Quantisation is what makes home inference possible at all: at 4-bit a weight takes a little over half a byte, so a 9B model that needs 18 GB as trained needs 5.5 GB, and the difference in what it writes is small enough that Q4_K_M is the default download for a reason. Below that the loss becomes visible on small models; above Q6 nobody can tell.

The context is the variable cost, and it is the one that gets you. Every token the model has read is kept as a key and a value in every attention layer, the KV cache, and that memory scales with the context length you allow, not the length you use. A dense 8B model keeps 128 KiB per token, so 32K of context is 4 GB, most of the room a 16 GB Mac has left after the weights. Newer architectures cut this hard: Qwen3.5 keeps a growing cache on only one layer in four and a fixed-size state on the rest, so it holds 32 KiB per token, a quarter of the dense figure at the same size. That is why the 9B fit the measured laptop with a long context and an older 8B would not have.

A Mac does not give the GPU the whole box. macOS caps the memory Metal may pin, at 12.7 GB of 16 on the measured machine, and buffers, activations and the engine itself come out of that before any weights do. So a 16 GB Mac is an 11 GB machine for this purpose. The 9B Q4 fit it, and left the rest of the computer 825 MB, which is what the warning above means: it runs, and everything else swaps. Shipping models are sized in whole gigabytes; the number to plan around is the one after the cap and the reserve.