You are reading immutable version 28. The current guide may be newer.

Study the paper

Attention Is All You Need

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Empirical Performance and Generalization

F1 scores on English constituency parsing

Interactive study view

F1 scores on English constituency parsing

Paper evidence + deterministic derivation

Explore paper-reported results and reconciled deterministic comparisons.

F1 scores on English constituency parsing data
labellabelsource valuelabelProvenance
Improvement of semi-supervised Transformer over discriminative TransformerDerived comparison1.4F1Deterministic derivation
Transformer (4 layers) semi-supervised F1F192.7F1Paper evidence
Transformer (4 layers) WSJ only, discriminative F1F191.3F1Paper evidence