You are reading immutable version 38. The current guide may be newer.

Study the paper

Attention Is All You Need

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Scaled dot-product attention

How attention redistributes weight

Interactive study view

How attention redistributes weight

Illustrative teaching data

Illustrative attention weights computed from a fixed teaching example; these are not paper-reported values.

Illustrative teaching data

Values in attention probability; scale 0 to 1
From / toToken 1Token 2Token 3Token 4
Token 1
Token 2
Token 3
Token 4