Study the paper
Attention Is All You Need
Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.
Integrated assessment
Research and implementation context
Supplementary context
Research and implementation context
Research lineage
Notable related work, not a complete survey
Predecessors
Deep Residual Learning for Image Recognition
Selected from the bounded candidate set.
Adam: A Method for Stochastic Optimization
Selected from the bounded candidate set.
Long Short-Term Memory
Selected from the bounded candidate set.
Successors
Weaver: Kronecker product approximations of spatiotemporal attention for traffic network forecasting
Selected from the bounded candidate set.
TDFormer: A novel triple decoupled transformer for accurate multi-step wind power forecasting
Selected from the bounded candidate set.
Automatic prompt generation via reinforcement learning guided contrastive purification for label-free SAM segmentation
Selected from the bounded candidate set.
Implementation code map
Static paths for orientation only. DeepStudy does not clone, install, build, or execute repository content.
Official repository
tensorflow/tensor2tensor
Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.
- .travis.ymlHigh confidence
Static high-confidence entry point identified from the default-branch tree.
- floyd.ymlHigh confidence
Static high-confidence entry point identified from the default-branch tree.
- README.mdHigh confidence
Static high-confidence entry point identified from the default-branch tree.
- .gitignoreLow confidence
Static low-confidence entry point identified from the default-branch tree.
- AUTHORSLow confidence
Static low-confidence entry point identified from the default-branch tree.
- CONTRIBUTING.mdLow confidence
Static low-confidence entry point identified from the default-branch tree.
- floyd_requirements.txtLow confidence
Static low-confidence entry point identified from the default-branch tree.
- ISSUE_TEMPLATE.mdLow confidence
Static low-confidence entry point identified from the default-branch tree.
- LICENSELow confidence
Static low-confidence entry point identified from the default-branch tree.
- pylintrcLow confidence
Static low-confidence entry point identified from the default-branch tree.
- setup.pyLow confidence
Static low-confidence entry point identified from the default-branch tree.
- .travis.ymlHigh confidence
Community repository
AyoOdumark/Attention-is-all-you-need
https://https://arxiv.org/abs/1706.03762
- src/__init__.pyMedium confidence
Static medium-confidence entry point identified from the default-branch tree.
- src/attention.pyMedium confidence
Static medium-confidence entry point identified from the default-branch tree.
- src/tokenization.pyMedium confidence
Static medium-confidence entry point identified from the default-branch tree.
- src/transformer.pyMedium confidence
Static medium-confidence entry point identified from the default-branch tree.
- src/__init__.pyMedium confidence