Attention degradation
The loss of focus that can occur as relevant details compete with a growing amount of context.
Illustrative conversation
- The requirement is in the prompt, so why did the model ignore it?
- That is one possible cause. Check whether it was supplied, conflicted with other instructions, or was lost during compaction before blaming length.
Definition
Attention degradation describes reduced accuracy or focus as context becomes longer, noisier, or poorly arranged. Research has shown that a model may use evidence differently depending on where it appears, with information in the middle of long contexts sometimes harder to retrieve than information near the edges. It is a performance gradient, not a universal token threshold, and varies by model and task.
Reviewed