← Back to Basic Science

Backpropagation

1986
Computer Science (theoretical)Algorithms & Data StructuresAlgorithmfoundational

Efficient reverse-mode computation of gradients through a layered model, anchored at Rumelhart, Hinton & Williams ("Learning representations by back-propagating errors," Nature 323, 1986), whose specific contribution was the first empirical demonstration that backprop-trained hidden layers learn useful internal representations -- not just wider distribution of an existing algorithm. Rumelhart et al. independently re-derived the underlying technique without citing prior work. Split (2026-09-03) from its general mathematical ancestor, Reverse-Mode Automatic Differentiation (Linnainmaa, 1970, parent_node_id above) -- per BASIC_METHODOLOGY.md Tier 1 SS4 Option B, now that Basic has scaled past the 3-node POC that originally justified a single compromise-dated node (Option A). Werbos (1974 PhD thesis; concrete application in 1982) was arguably first to apply the technique to a neural-network-style model -- left as historical context here rather than modeled as a third node, since that claim is gradual/multi-source rather than one clean citable result to split off.

Discovery Velocity

Normalized OpenAlex paper velocity, one point per year.

Originators

  • Rumelhart, D.E.
  • Hinton, G.E.
  • Williams, R.J.

Landmark Paper

W1498436455 ↗
Not retracted (OpenAlex)

Checked 2026-08-09 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10

Connected Papers

Came before

  • Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences

    Paul Werbos · 1974

    W1573503290 ↗

    PhD thesis arguably first to apply reverse-mode differentiation to a neural-network-style model, though not published or widely cited at the time.

  • Applications of advances in nonlinear sensitivity analysis

    Paul Werbos · 2005

    W20283819 ↗

    Concrete published application of the 1974 method, still years before Rumelhart, Hinton & Williams independently re-derived and popularized it in 1986.

This discovery

Backpropagation

Rumelhart, D.E., Hinton, G.E., Williams, R.J. · 1986

W1498436455 ↗

Came after

No direct successor papers recorded yet

Connections

  • is prerequisite for LSTM
    basis: reasoned

    Backpropagation-through-time is the mechanism by which recurrent networks, including LSTM, are trained; Hochreiter, Bengio, Frasconi & Schmidhuber's analysis of vanishing/exploding gradients in backprop-through-time (ref [12] in Vaswani et al. 2017) is the specific problem LSTM's gating was designed to solve.