feed/tags/disaggregation
disaggregation
One explainer you can run, one glossary term, and 3 readings ordered as a path, each with one line on why it earns your time.
advanced/ 3 entries
Current work, at the depth practitioners actually argue about.
- toolNIXL: NVIDIA Inference Xfer Library
Disaggregation only pays if the KV cache moves between pools fast enough. This is the transfer layer underneath Dynamo doing that job.
#disaggregation#kv-cache#networking - articleThe case for disaggregated LLM serving
Disaggregation is usually argued for on principle. This argues it on conditions (scale, network, and traffic balance) and is honest that below them it does not pay.
Fergus Finn#disaggregation#prefill#decode#scale - paperSplitwise: Efficient Generative LLM Inference Using Phase Splitting
Prefill and decode want different hardware. The clearest statement of why disaggregation keeps winning.
#disaggregation#prefill#decode