feed/tags/agents
agents
One explainer you can run, 2 glossary terms, and 2 readings ordered as a path, each with one line on why it earns your time.
advanced/ 2 entries
Current work, at the depth practitioners actually argue about.
- articleAgentX and InferenceX v3: does the CUDA moat hold up in agentic inference?
An agent turn is 88K tokens in and 413 out, so the benchmark is really a KV cache hit-rate benchmark: 91% on B300 at 384 sessions, 73% on B200 with the rest spilling to DRAM. Nvidia, AMD, five open models, 70 upstream PRs. Partly paywalled.
Cam Quilici, Bryan Shan, Alec Ibarra and others#agents#prefix-cache#kv-cache#benchmark#hardware - articleThunderAgent: 2x Faster Agentic Inference for Synthetic Data Generation at Scale
An agent that pauses for a tool call gets its cache evicted and pays to rebuild it. Scheduling the whole program instead of each request is the fix.
Hao Kang and colleagues#agents#kv-cache#scheduling