TL;DR

Plain stacks of layers get harder to optimize as they get deeper. He et al. reformulate each block so it learns a residual: the difference from its own input, added back with a skip connection. Those residual networks train at depths that previously stalled, and the extra depth improves ImageNet accuracy.

Why it matters

Before this paper, adding layers was an unreliable way to make a vision model better. Training error could rise even when a deeper net should have been able to copy a shallower one. Residual learning made depth a practical dial for image recognition, and the same skip-connection idea later showed up throughout modern networks, including Transformers.

Key takeaways

• A residual block learns a correction and adds it to the input, instead of replacing the input outright.
• The shortcut path gives gradients a direct route through many layers.
• On ImageNet, residual nets with far more layers optimized more easily than plain counterparts.
• The paper's evidence is that depth helps once the optimization problem is reformulated, not that depth is free.