Backpropagation
Efficient gradient computation that lets multilayer networks learn internal representations; the training algorithm behind virtually all deep learning.
- 1986

First version
David Rumelhart, Geoffrey Hinton, Ronald Williams (popularised); Seppo Linnainmaa, Paul Werbos (earlier)
Image: Carnegie Mellon University · © Carnegie Mellon University; university seal (trademark), no licence stated · Carnegie Mellon University brand guidelines, 'University Seal' page (one-colour black seal sample) - 1989

Backprop-trained CNNs
LeCun applies backprop to convolutional nets for real-world digit recognition.
Image: Petri Kutvonen (University of Helsinki), photo c. 1988 · Unclear; no licence stated (© Petri Kut… · Petri Kutvonen's photo pages, University of Hel… - 1997

LSTM (Hochreiter & Schmidhuber)
Backprop through time made workable for long sequences via gated memory cells.
Image: IDSIA (Istituto Dalle Molle di Studi sull'Intelligenza Artificiale) · Unclear; no licence stated (institute l… · Wayback Machine capture of www.idsia.ch, 16 Jul… - 2015

Automatic-differentiation frameworks
TensorFlow (2015) and PyTorch (2016) make backprop automatic for any model.
Image: TensorFlow - vectors combined, edited - Begoon · Apache License 2.0 · Wikimedia Commons
Contested: Linnainmaa published reverse-mode differentiation in 1970 and Werbos proposed it for neural nets in 1974/1982; the 1986 Nature paper popularised it.