Study the paper

Attention Is All You Need

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Scaled Dot-Product Attention

How attention redistributes weight

Interactive study view

How attention redistributes weight

Illustrative teaching data

Illustrative attention weights computed from a fixed teaching example; these are not paper-reported values.

Illustrative teaching data

Values in attention probability; scale 0 to 1
From / toToken 1Token 2Token 3Token 4
Token 1
Token 2
Token 3
Token 4