Why recall dies as context grows

The advertised window is capacity. The effective window is where attention still works. Filling more of a 128k window does not make the model “see more” — it thins the attention budget, and the middle of the prompt is where that budget runs out first.
advertised window — capacity start middle — the hole end effective recall lives at the edges
Place the fact · grow the haystack · change the task
■ attention mass (brighter = more)■ the fact
Recall @ this position
—
If fact at start
—
If fact at middle
—
Effective vs advertised
—
What to try
  • Leave the fact at 50% (middle). Drag haystack 4k → 64k. Needle recall droops; switch to 2-hop and it falls off a cliff. Bigger window, worse middle.
  • At 32k, slide the fact 0% → 50% → 100%. U-shape: start and end survive; middle is the hole. That is Lost-in-the-Middle, not “the window filled.”
  • Needle at 32k middle can still look “ok.” 2-hop at the same place is the agent failure (constraint in turn 12).
Ask first: advertised window, or effective? Softmax is a budget. More tokens thin every key; the middle pays first. Pin at the start. Don’t keep the haystack. DSA (2512.02556) can skip looking at the middle; those tokens are often still stored, and the U-curve still applies to whatever the indexer picked. Compaction is how you stop storing the haystack. Split look vs store in the next lab.