Skip to content
forward pass

Gemini

Google merges Brain and DeepMind and releases a model trained from the start on text, images, audio and video together; the search company catches up to GPT-4.

category
model
significance
3 of 5
people
Demis Hassabis, Sundar Pichai, Oriol Vinyals
organisations
Google DeepMind

what had to happen · 53 events back to 1943

Every event this one built on, transitively, in order. The organism has the same path lit. Direct influences are marked.

00 · One neuron · 1

  1. 1943A logical calculus of nervous activity

I · Foundations · 11

  1. 1948A mathematical theory of communication
  2. 1949Cells that fire together wire together
  3. 1950Programming a computer for playing chess
  4. 1950Computing machinery and intelligence
  5. 1958The perceptron learns
  6. 1959Samuel's checkers program coins 'machine learning'
  7. 1960ADALINE and the least-mean-squares rule
  8. 1965Moore's law
  9. 1966ELIZA
  10. 1969Perceptrons
  11. 1970Reverse-mode automatic differentiation

W1 · The first winter · 2

  1. 1974Werbos applies backpropagation to neural networks
  2. 1980The Neocognitron

II · Connection · 3

  1. 1982The Hopfield network
  2. 1985The Boltzmann machine
  3. 1986Backpropagation

W2 · The second winter · 6

  1. 1988Temporal-difference learning
  2. 1989Q-learning
  3. 1989LeNet reads handwritten postcodes
  4. 1990Finding structure in time
  5. 1991The vanishing gradient problem
  6. 1992TD-Gammon reaches world-class backgammon

III · Statistics and data · 9

  1. 1997Long short-term memory
  2. 1998MNIST and LeNet-5
  3. 1999The first GPU
  4. 2003A neural probabilistic language model
  5. 2006Deep belief networks and the word 'deep'
  6. 2007CUDA
  7. 2009Deep learning moves to GPUs
  8. 2009ImageNet
  9. 2010Rectified linear units

IV · Deep learning · 9

  1. 2012Google Brain's network discovers cats
  2. 2012Dropout
  3. 2012AlexNet wins ImageNet
  4. 2013Deep Q-networks play Atari
  5. 2014Attention
  6. 2014Sequence to sequence learning
  7. 2015Batch normalisation
  8. 2015Residual networks
  9. 2016Google reveals the TPU

V · Transformers · 7

  1. 2017Attention is all you need
  2. 2017Deep reinforcement learning from human preferences
  3. 2018GPT: generative pre-training
  4. 2019GPT-2 and the model too dangerous to release
  5. 2020Scaling laws for neural language models
  6. 2020GPT-3
  7. 2020Learning to summarise from human feedback

VI · Everyone · 5

  1. 2022InstructGPT
  2. 2022Chain-of-thought prompting
  3. 2022PaLMdirect
  4. 2022ChatGPT
  5. 2023GPT-4direct

Google had invented the transformer, run the largest models in the world and been caught flat-footed by ChatGPT. In April 2023 it merged Google Brain and DeepMind into one organisation under Demis Hassabis, and on 6 December it released the result. Gemini came in three sizes, Ultra, Pro and Nano, was trained on TPU v4 and v5 pods, and was natively multimodal: rather than bolting an image encoder onto a language model, it had been trained from the beginning on interleaved text, code, images, audio and video. Ultra was the first model to beat human experts on the MMLU benchmark, at 90 percent, and matched or exceeded GPT-4 on most others.

The launch also included a demonstration video that had been edited to look like real-time interaction, and the criticism it drew set the tone for a year in which Google's models were judged harshly. Gemini 1.5 in February 2024 introduced a million-token context window, which changed what the models could be asked to read.

Gemini is on this timeline as the moment the three-laboratory frontier settled, OpenAI, Anthropic and Google DeepMind, and as the model family that in 2025 achieved gold-medal performance at the mathematical olympiad and, as Gemini 3, took the lead on most benchmarks.

what it led to · 1 events downstream, through 2025

Built on it directly:

  1. 2025Gemini 3VII

sources · 1

Trace the lineage of this event on the timeline →Back to the ledger