Attention relationship

Also called: token relationship, attention pair, attention weights, attention score, attention scores

The link between two tokens in the input, measured by how much one shapes the model's reading of the other. Every token can relate to every other one, so these links multiply fast: double the text and you get roughly four times as many.

For example: In a long contract, a clause on page 9 that mentions 'the fee above' has an attention relationship with the fee set out on page 2.

Connects to 5

Attention Concept
Token Concept
Attention budget Concept
Attention degradation Concept
Context window Concept