Study the paper
Deep Residual Learning for Image Recognition
Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.
Course Overview: Deep Residual Learning
At a glance
At a glance
Course Overview: Deep Residual Learning
Sources
section
1 Introduction
As neural networks grow deeper, they encounter a critical optimization barrier known as the degradation problem. Unlike traditional issues like overfitting, degradation manifests as a drop in training accuracy when more layers are added to a sufficiently deep model. This course explores the residual learning framework designed to overcome this barrier, enabling the successful training of networks with hundreds or thousands of layers.
Sources
S1.p3.1
When deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly. Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments. Fig. 1 shows a typical example.