Study the paper
Attention Is All You Need
Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.
8 activities
At a glance
Why attention replaces recurrenceThe Transformer replaces recurrent sequence processing with attention-driven computation.
LessonWhy replace recurrence?
Why attention replaces recurrenceThe paper introduces an architecture that replaces recurrent sequence processing with attention-driven computation.
LessonEncoder-decoder organization
Encoder-decoder architectureThe model is organized around an encoder and a decoder, the two major structural parts named in the source excerpt.
LessonScaled dot-product attention
Scaled dot-product attentionQueries compare with keys, and the resulting weights combine values.
LessonScaled dot-product attention
Scaled dot-product attentionmathrm Attention (Q,K,V)= mathrm softmax ( frac QK^ T sqrt d k )V
LessonScope of the supplied evidence
Evidence and limitationsThe supplied passages ground the recurrence-replacement claim and the encoder-decoder structure. Claims beyond that scope require additional source evidence.
LessonEvidence available in the supplied excerpt
Evidence and limitationsThe supplied excerpt identifies the introduction section but contains no numerical translation result. The missing value is preserved rather than inferred.
QuizTest your understanding
Integrated assessment10 questions grounded in this paper section.