Statistical machine learning (decision trees, SVMs, ensembles)
Algorithms that learn prediction rules from labelled examples; they ran most practical machine learning before deep learning and still lead on table-shaped business data.
- 1984

First version
Leo Breiman, Jerome Friedman, Richard Olshen and Charles Stone (CART); Ross Quinlan (ID3); Vladimir Vapnik and colleagues (SVM, Bell Labs)
Image: Retro-Computing Society of Rhode Island · CC BY-SA 3.0 · Wikimedia Commons - 1992

Support-vector machines with kernels
Boser, Guyon and Vapnik combine maximum-margin classifiers with the kernel trick to learn non-linear boundaries with strong guarantees.
Image: Bell Labs · Public domain · Wikimedia Commons - 1997

Boosting (AdaBoost)
Freund and Schapire show many weak learners can be combined into one highly accurate model.
Image: AT&T Corp. (AT&T Laboratories) · Unclear; no licence stated (company log… · Wayback Machine capture (27 Jul 1997) of www.re… - 2001

Random forests
Breiman averages many randomised decision trees into a robust, near tuning-free model.
Image: University of California, Berkeley · Unclear; institutional logo, no licence… · Wayback Machine capture of www.berkeley.edu, 6 … - 2016

XGBoost (scalable gradient boosting)
A fast, open-source gradient-boosted tree system that dominates competitions and industry work on tabular data.
Image: Hcho3wiki · CC BY-SA 4.0 · Wikimedia Commons
Contested: Tree learners predate the 1984 CART book: the THAID program (1972) built classification trees, CART circulated as reports from 1977, and Quinlan's ID3 work ran in parallel (published 1986).