Residual Connections
2015Identity shortcut connections that let gradients skip layers directly, addressing the degradation problem in very deep networks and enabling training of substantially deeper models (He, Zhang, Ren & Sun, "Deep Residual Learning for Image Recognition," CVPR 2016). Originated in image recognition; reused structurally as a sub-layer wrapper in later sequence-transduction architectures.
Discovery Velocity
Normalized OpenAlex paper velocity, one point per year.
Originators
- He, K.
- Zhang, X.
- Ren, S.
- Sun, J.
Landmark Paper
Checked 2026-09-19 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10
Connections
- is component of Transformerbasis: reasoned
Section 3.1 of Vaswani et al. 2017 wraps every sub-layer in a residual connection (ref [11], He et al.), "LayerNorm(x + Sublayer(x))", citing He et al.'s residual learning directly as the mechanism enabling the full 6-layer encoder/decoder stack to train.
Cross-Domain Connections
No verified cross-domain connections indexed for Residual Connections yet. Verified links are admitted under FSTP epistemic audit standards.