Figure 2 — The cache boundary: stable prefix vs. tool loop
⬡
System
Prompt
💬
History
msg 1
💬
History
msg 2
💬
History
msg 3
completedPrefixLength
🔧
Tool
Result
iter 1
✦
Assistant
iter 1
🔧
Tool
Result
iter 2
✦
Assistant
iter 2
Stable Prefix — cached
Paid once per cache write. Reads cost 0.1×.
↻
Tool Loop — uncached
New each iteration. Full rate on every call.
Legend
System prompt
History messages
Tool result
Assistant
Cache boundary