Study the paper

Attention Is All You Need

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

Evidence and limitations

Scope of the supplied evidence

Deep dive

The supplied passages ground the recurrence-replacement claim and the encoder-decoder structure. Claims beyond that scope require additional source evidence.

Sources

section

1 Introduction

S1.p3.1

Attention mechanisms have become an integral part of compelling sequence modeling and transduction models in various tasks, allowing modeling of dependencies without regard to their distance in the input or output sequences [2, 19]. In all but a few cases [27], however, such attention mechanisms are used in conjunction with a recurrent network.