Backpropagation
1986Efficient reverse-mode computation of gradients through a layered model, anchored at Rumelhart, Hinton & Williams ("Learning representations by back-propagating errors," Nature 323, 1986), whose specific contribution was the first empirical demonstration that backprop-trained hidden layers learn useful internal representations -- not just wider distribution of an existing algorithm. Rumelhart et al. independently re-derived the underlying technique without citing prior work. Split (2026-09-03) from its general mathematical ancestor, Reverse-Mode Automatic Differentiation (Linnainmaa, 1970, parent_node_id above) -- per BASIC_METHODOLOGY.md Tier 1 SS4 Option B, now that Basic has scaled past the 3-node POC that originally justified a single compromise-dated node (Option A). Werbos (1974 PhD thesis; concrete application in 1982) was arguably first to apply the technique to a neural-network-style model -- left as historical context here rather than modeled as a third node, since that claim is gradual/multi-source rather than one clean citable result to split off.
Discovery Velocity
Normalized OpenAlex paper velocity, one point per year.
Originators
- Rumelhart, D.E.
- Hinton, G.E.
- Williams, R.J.
Landmark Paper
Checked 2026-08-09 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10
Connected Papers
Came before
Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences
Paul Werbos · 1974
W1573503290 ↗PhD thesis arguably first to apply reverse-mode differentiation to a neural-network-style model, though not published or widely cited at the time.
Applications of advances in nonlinear sensitivity analysis
Paul Werbos · 2005
W20283819 ↗Concrete published application of the 1974 method, still years before Rumelhart, Hinton & Williams independently re-derived and popularized it in 1986.
Came after
No direct successor papers recorded yet
Lineage
Refinement of Reverse-Mode Automatic Differentiation
Connections
- is prerequisite for LSTMbasis: reasoned
Backpropagation-through-time is the mechanism by which recurrent networks, including LSTM, are trained; Hochreiter, Bengio, Frasconi & Schmidhuber's analysis of vanishing/exploding gradients in backprop-through-time (ref [12] in Vaswani et al. 2017) is the specific problem LSTM's gating was designed to solve.
Bidirectional Cross-Domain Connections
Traverse causal and structural relationships bridging basic scientific foundations, applied technology lineages, and commercial company milestones.
Gradient-Based Representation Learning
Backpropagation error gradients enable dual-weight sharing architectures in Siamese Networks to learn metric embeddings.
Neural Function Approximation for RL
Backpropagation allowed reinforcement learning Bellman updates to scale from tabular states to deep neural network approximations.
Acquisition of Frontier Deep Learning Lab
Google acquired DeepMind following their Nature Atari breakthrough applying deep Q-learning directly from pixels.