You are reading immutable version 1. The current guide may be newer.

Study the paper

Deep Residual Learning for Image Recognition

Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.

All activities

Comprehensive Assessment

Test your understanding

Check your understanding

1. Why is the degradation problem in deep networks distinguished from overfitting?
2. Explain the 'solution by construction' argument and how it demonstrates that deeper networks should theoretically perform at least as well as shallower ones.
3. If the desired underlying mapping to be fit by a stack of layers is $\mathcal{H}(\mathbf{x})$, how is the residual mapping $\mathcal{F}(\mathbf{x})$ defined?
4. Why is it hypothesized that optimizing the residual mapping $\mathcal{F}(\mathbf{x})$ is easier for a solver than directly fitting the original mapping $\mathcal{H}(\mathbf{x})$?
5. What is the impact of identity shortcut connections on the parameter count and computational complexity of a neural network?
6. Write down the mathematical formulation of a residual block with identity shortcut connections as defined in Equation (1), and define each variable.
7. Under what circumstances must a projection shortcut (using $W_s$) be used instead of a parameter-free identity shortcut?
8. Compare the three options (A, B, and C) evaluated in the paper for handling dimension changes, and identify which option uses projection shortcuts for all connections.
9. What are the filter sizes of the three convolutional layers in the bottleneck building block?
10. Explain why replacing identity shortcuts with projection shortcuts in a bottleneck architecture significantly increases the model size and time complexity.
11. What does the analysis of standard deviations of layer responses reveal about ResNets compared to plain networks?
12. How does the magnitude of layer responses change as the depth of the ResNet increases?