Attention, please!

A transformer reads a sentence by letting every word look at every other word and decide how much of each one to pull into its own meaning. Those weights are called attention, and you can read them straight out of the model.

Type something below and a small model will show you where each word looks. It runs in your browser, so nothing you type is sent anywhere. This is a pared-down take on Attention on moebio.com, which asks a language model to score every pair of words. Here the numbers are the model's own weights, and there is only the matrix.

Your text

0 / 280

Layer Head

The matrix is wider than your screen; scroll it sideways.

Each row is a word doing the looking, and each column is a word being looked at. A row adds up to 100%, and its cells are shaded against that row's strongest link, so every row has one full-colour cell.

The example opens on layer 4, head 2, because that is where both “it”s land on their nouns: the first looks hardest at “cup” and the second at “robot”. That is one head out of 72, picked because it looks good. Most of the others are busy with the next word, or with the word itself.

Limitations

Attention is not understanding. A heavy weight means the model pulled information from that word at that step. It does not mean the model grasped the relationship between the two, and plenty of heads just look at the next word or at the word itself.
It is a small model. MiniLM has 6 layers, 12 heads per layer and about 22 million parameters, and it was trained to judge whether sentences mean the same thing, not to resolve pronouns. A large language model spreads its attention differently.
The weights are compressed. To make the download four times smaller, the model is stored at 8-bit precision. The average cell moves by less than half a percentage point, but a few move a lot, and about one word in ten ends up with a different strongest link than in the full-precision model.
Words, not tokens. The model reads word pieces (“kitchen,” is “kitchen” then “,”). Pieces are merged back into words: a row averages its pieces, and a column adds them up. The model also sends attention to two marker tokens that open and close the text: in the middle layers close to half of it on average, and some heads send nearly all of it there. Those are hidden, and each row is rescaled over the words that are left.
English, 280 characters. The vocabulary is lowercase English, so emoji and most other scripts come through as “unknown”.

Model: all-MiniLM-L6-v2, in Qdrant's export that returns its attention weights, run with Transformers.js. The first run downloads about 23 MB; after that it comes from your browser's cache.