Study the paper

Deep Residual Learning for Image Recognition

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Comprehensive Assessment

Test your understanding

Check your understanding

1. Which of the following statements best distinguishes the 'degradation problem' from overfitting in deep neural networks?
2. Explain the 'solution by construction' argument used to show that a deeper model should theoretically perform no worse than its shallower counterpart on training data.
3. If H(x) is the desired underlying mapping to be fit by a few stacked layers, how is the residual mapping F(x) formulated?
4. Why is optimizing the residual mapping F(x) hypothesized to be easier than optimizing the original unreferenced mapping H(x), particularly in the extreme case where an identity mapping is optimal?
5. What is a key advantage of using identity shortcut connections in residual networks?
6. Write the mathematical equation for a residual block with an identity shortcut connection, where x is the input vector, y is the output vector, and F is the residual function to be learned.
7. When matching dimensions in shortcuts where the output dimension is larger than the input dimension, which option introduces extra parameters?
8. Write the mathematical equation for a residual block that utilizes a projection shortcut to match dimensions.
9. What is the sequence of convolutional layers used in the 'bottleneck' building block for deeper ResNets?
10. Explain why parameter-free identity shortcuts are particularly important for the bottleneck architectures compared to projection shortcuts.
11. What does the analysis of standard deviations of layer responses reveal about ResNets compared to plain networks?
12. How does increasing the depth of a ResNet (e.g., from 20 to 110 layers) affect the magnitude of individual layer responses?
13. Why did the 1202-layer ResNet perform worse on the test set than the 110-layer ResNet on CIFAR-10, despite having similar training error?
14. What regularization approach did the authors take for the 1202-layer network on CIFAR-10, and what do they suggest for future improvement?