Backpropagation
1986Efficient reverse-mode computation of gradients through a layered model, anchored at Rumelhart, Hinton & Williams ("Learning representations by back-propagating errors," Nature 323, 1986), whose specific contribution was the first empirical demonstration that backprop-trained hidden layers learn useful internal representations -- not just wider distribution of an existing algorithm. Rumelhart et al. independently re-derived the underlying technique without citing prior work. Split (2026-09-03) from its general mathematical ancestor, Reverse-Mode Automatic Differentiation (Linnainmaa, 1970, parent_node_id above) -- per BASIC_METHODOLOGY.md Tier 1 SS4 Option B, now that Basic has scaled past the 3-node POC that originally justified a single compromise-dated node (Option A). Werbos (1974 PhD thesis; concrete application in 1982) was arguably first to apply the technique to a neural-network-style model -- left as historical context here rather than modeled as a third node, since that claim is gradual/multi-source rather than one clean citable result to split off.
Discovery Velocity
Normalized OpenAlex paper velocity, one point per year.
Originators
- Rumelhart, D.E.
- Hinton, G.E.
- Williams, R.J.
Landmark Paper
Checked 2026-08-09 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10
Connected Papers
Came before
Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences
Paul Werbos · 1974
W1573503290 ↗PhD thesis arguably first to apply reverse-mode differentiation to a neural-network-style model, though not published or widely cited at the time.
Applications of advances in nonlinear sensitivity analysis
Paul Werbos · 2005
W20283819 ↗Concrete published application of the 1974 method, still years before Rumelhart, Hinton & Williams independently re-derived and popularized it in 1986.
Came after
No direct successor papers recorded yet
Lineage
Refinement of Reverse-Mode Automatic Differentiation
Connections
- is prerequisite for LSTMbasis: reasoned
Backpropagation-through-time is the mechanism by which recurrent networks, including LSTM, are trained; Hochreiter, Bengio, Frasconi & Schmidhuber's analysis of vanishing/exploding gradients in backprop-through-time (ref [12] in Vaswani et al. 2017) is the specific problem LSTM's gating was designed to solve.