Explainers
The concepts serving actually turns on, as things you can operate. Every page here is a simulation with the inputs exposed. Change them and watch the system misbehave, which is the part a diagram cannot show you.
explainers/
- beginnerA batch is only as fast as its slowest member
Run the same fourteen requests under static and continuous batching, and watch where a GPU's idle time actually comes from.
- beginnerThe KV cache is the thing you are actually renting
Fill a GPU with weights and per-request cache until it will not take another user. 320 KiB per token adds up faster than anyone expects.
- intermediateTwo phases, two bottlenecks(not built yet)
Move sequence length and batch size across a roofline and watch the limit flip from compute to memory bandwidth, the reason disaggregation exists.
- intermediateLatency, cost, quality: pick two(not built yet)
Drag a point inside the triangle and see which real configurations land there, priced from the nightly catalogue.