Study the paper
Deep Residual Learning for Image Recognition
Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.
8 activities
At a glance
Course Overview: Deep Residual LearningCourse Overview: Deep Residual Learning
LessonThe Degradation Problem and Optimization Difficulties
The Degradation Problem and Optimization DifficultiesUnderstanding the Degradation Problem
LessonThe Degradation Problem and Residual Learning Context
The Degradation Problem and Optimization DifficultiesThe Degradation Problem in Deep Networks
LessonResidual Block with Identity Shortcut
Mathematical Formulation of Residual LearningThis equation defines the fundamental residual block with an identity shortcut connection. The input vector is directly added to the output of the residual function, allowing gradients to flow unimpeded during backpropagation.
LessonResidual Block with Projection Shortcut
Identity vs. Projection ShortcutsWhen the dimensions of the input mathbf x and the residual output mathcal F differ (for example, when changing channel depths), a projection shortcut W s is applied to the input to match the dimensions.
LessonDeeper Bottleneck Architectures Walkthrough
Deeper Bottleneck ArchitecturesDeeper Bottleneck Architectures
LessonStandard deviations of layer responses on CIFAR-10
Empirical Analysis of Layer ResponsesAn empirical analysis of the standard deviations (std) of layer responses on CIFAR-10 was conducted to investigate the preconditioning hypothesis. The responses are measured at the outputs of each 3 times3 layer, after Batch Normalization (BN) and before nonlinearity.
QuizTest your understanding
Comprehensive Assessment12 questions grounded in this paper section.