Study the paper

Attention Is All You Need

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Multi-Head Attention Mechanics

Multi-Head Attention: step by step

Interactive study view

Multi-Head Attention: step by step

Illustrative teaching data

Follow the existing bounded walkthrough in its intended sequence.

Step 1 of 3

  1. Step 1

    Concatenate head_1 and head_2 to get [1.0, 2.0, 3.0, 4.0]

  2. Step 2

    Multiply by the projection matrix W^O to obtain the final output vector [4.0, 6.0]

  3. Step 3

    Sum the elements of the output vector to produce a single scalar result: 4.0 + 6.0 = 10.0