← Back to Basic Science

Adam

2014
MathematicsOptimization & Control TheoryAlgorithmfoundational

Adaptive moment estimation optimizer (Kingma & Ba, arXiv:1412.6980, ICLR 2015) -- refines SGD with per-parameter adaptive learning rates via first/second moment estimates. Same stochastic-approximation family as its parent, not a new field.

Discovery Velocity

Normalized OpenAlex paper velocity, one point per year.

Originators

  • Kingma, D.P.
  • Ba, J.

Landmark Paper

W1522301498 ↗
Not retracted (OpenAlex)

Checked 2026-08-09 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10

Lineage

Refinement of Stochastic Gradient Descent

Connections

  • is component of Transformer
    basis: reasoned

    Section 5.3 of Vaswani et al. 2017 specifies the Adam optimizer (ref [20], beta1=0.9, beta2=0.98, epsilon=1e-9) with a custom warmup / inverse-square-root learning-rate schedule as the training procedure used for every reported result.