Large language models (GPT lineage and rivals)

Transformers pretrained on vast text that generate language, code and reasoning; the general-purpose engine of the 2020s AI boom.

AI & roboticsSoftwareLandmark
  1. 2018

    First version

    OpenAI (Alec Radford et al.); later Anthropic, Google DeepMind, Meta, DeepSeek and others

    Image: OpenAI · Public domain · Wikimedia Commons
  2. 2019

    GPT-2

    Tenfold scale-up (1.5B parameters) writes coherent long text and performs tasks without task-specific training.

    Image: OpenAI · Public domain · Wikimedia Commons
  3. 2020

    GPT-3

    175B parameters; few-shot learning from prompts without fine-tuning.

    Image: OpenAI · Public domain · Wikimedia Commons
  4. 2022

    ChatGPT (instruction-tuned with RLHF)

    Human-feedback training turns the raw model into an assistant that follows instructions in conversation.

    Image: OpenAI · Public domain · Wikimedia Commons
  5. 2023

    GPT-4

    Multimodal: accepts images as well as text and scores near top human levels on professional exams.

    Image: OpenAI · Public domain · Wikimedia Commons
  6. 2024

    Reasoning models (OpenAI o1)

    RL-trained chain-of-thought spends more compute 'thinking' before answering.

    Image: OpenAI · © OpenAI (official launch artwork; no r… · OpenAI o1 launch page, 12 Sep 2024 (Wayback cap…

Contested: ULMFiT and ELMo (2018) also pioneered pretrained language models; what counts as 'large' has no agreed threshold.

See it on the timeline