Study the paper
Deep Residual Learning for Image Recognition
Lessons, visuals, quizzes, flashcards, and resources—organized in teaching order.
Course Overview: Deep Residual Learning
At a glance
At a glance
Course Overview: Deep Residual Learning
Sources
section
1 Introduction
In deep learning, increasing network depth has been a primary driver of performance. However, as networks become deeper, a degradation problem emerges: beyond a certain depth, accuracy saturates and then degrades rapidly. This degradation is not caused by overfitting, as it manifests as higher training error alongside higher test error.
Sources
S1.p3.1
When deeper networks are able to start converging, a degradation problem has been exposed: with the network depth increasing, accuracy gets saturated (which might be unsurprising) and then degrades rapidly. Unexpectedly, such degradation is not caused by overfitting, and adding more layers to a suitably deep model leads to higher training error, as reported in [11, 42] and thoroughly verified by our experiments. Fig. 1 shows a typical example.