Attention
The mechanism that lets a language model weigh which parts of its input matter most for generating each output token. The core innovation of the transformer architecture.
Attention is the mechanism inside a transformer that lets a model decide, for each token it generates, which parts of the input to focus on. It's why models can attend to a specific earlier passage even when it's thousands of tokens away.
Mechanically, it's a weighted average. For each output position, the model computes an attention score between that position and every other position in the input. Higher-scoring positions contribute more. Every generated token gets its own pattern.
Attention comes from the 2017 paper "Attention is All You Need" — the Google research that introduced the transformer and displaced RNNs/LSTMs. Those ideas underpin GPT, Claude, Gemini, and essentially all modern language models.
Why it matters for users: - **Long-context recall**: attention lets a model refer to page 200 when answering about page 245. Modern models scale to 100K-1M+ tokens. - **"Lost in the middle"**: attention isn't uniform — content at the beginning and end of long inputs gets more weight than the middle. Critical info buried mid-doc can be missed. - **Expensive**: attention scales quadratically with input length. Efficient variants (flash, sparse) are active research areas.
Knowing this helps you write prompts: put load-bearing context at the beginning or end, not buried in the middle.