The history of large language models
65 milestones from the 2017 Transformer paper to this month's releases: research breakthroughs, model launches, open-weight releases and the rules that followed. Every date links to a primary source.
2017
- Jun 12, 2017
Vaswani et al. replaced recurrence with self-attention, enabling highly parallel training; nearly every later LLM builds on this architecture.
Google, Research
- Jun 12, 2017
Showed agents could learn complex goals from human comparisons of their behavior, the preference-learning idea later adapted to fine-tune language models with RLHF.
OpenAI / DeepMind, Research
2018
- Jun 11, 2018
OpenAI paired a Transformer with unsupervised pre-training on unlabeled text followed by task-specific fine-tuning, the basic recipe that later GPT models scaled up.
OpenAI, Research
- Oct 11, 2018
Bidirectional pre-training with masked-word prediction set new state-of-the-art results on eleven NLP tasks, popularizing fine-tuned encoder models for language understanding.
Google, Research
2019
- Feb 14, 2019
OpenAI's 1.5B-parameter model wrote fluent passages of text, and the company held back the full weights over misuse concerns, releasing larger versions in stages.
OpenAI, Model release
- Oct 23, 2019
T5 cast every NLP task as text in, text out, systematically compared pre-training choices, and released the C4 web corpus with its models.
Google, Research
2020
- Jan 23, 2020
Kaplan et al. found language-model loss falls predictably as a power law in parameters, data and compute, giving labs a basis for planning larger training runs.
OpenAI, Research
- May 22, 2020
Paired a text generator with a neural retriever over a document index, an approach now widely used to ground chatbots in current or private documents.
Facebook AI Research (Meta), Research
- May 28, 2020
At 175B parameters, GPT-3 could perform many tasks from a few examples in the prompt, popularizing in-context learning as an alternative to fine-tuning.
OpenAI, Model release
2021
- Jan 11, 2021
Routing each token to a single expert let Google train trillion-parameter sparse models efficiently, an early proof point for the mixture-of-experts designs common in later LLMs.
Google, Research
- Jun 17, 2021
Freezing pretrained weights and training small low-rank update matrices cut trainable parameters by 10,000x on GPT-3, making custom fine-tunes far cheaper.
Microsoft, Research
2022
- Jan 28, 2022
Wei et al. showed that prompting large models to write out intermediate steps markedly improved arithmetic, commonsense and symbolic reasoning, an idea later built into reasoning models.
Google, Research
- Mar 4, 2022
Fine-tuning GPT-3 on human demonstrations and preference rankings produced a 1.3B model that labelers preferred over the 175B original, a method ChatGPT later adopted.
OpenAI, Research
- Mar 29, 2022
DeepMind found large models were significantly undertrained; its 70B Chinchilla, trained on more data, outperformed far larger models and influenced how labs balance size against data.
DeepMind, Research
- Jul 12, 2022
Built by the BigScience research collaboration, the 176B-parameter model covered 46 natural and 13 programming languages and was openly released under a Responsible AI License.
BigScience / Hugging Face, Open weights
- Nov 30, 2022
A free research preview of a GPT-3.5-based assistant tuned with RLHF brought conversational AI to a mass audience; rival assistants from Google and Anthropic followed within months.
OpenAI, Product
- Dec 15, 2022
Anthropic trained a harmless assistant using AI feedback guided by a written set of principles, reducing reliance on human labels for safety training.
Anthropic, Research
2023
- Jan 23, 2023
A third phase of the partnership brought a multiyear, multibillion-dollar investment and more Azure supercomputing, showing how closely frontier labs depend on hyperscale cloud compute.
Microsoft / OpenAI, Policy
- Feb 6, 2023
Google opened its experimental Bard chatbot to trusted testers ahead of a public release and previewed AI features for Search; Bard was later renamed Gemini.
Google, Product
- Feb 24, 2023
Meta shared 7B to 65B models with approved researchers under a noncommercial license and reported that its 13B version outperformed 175B GPT-3 on most benchmarks.
Meta, Open weights
- Mar 14, 2023
OpenAI's multimodal model accepted images and text and scored around the top 10% on a simulated bar exam, versus the bottom 10% for GPT-3.5.
OpenAI, Model release
- Mar 14, 2023
After a closed alpha with partners such as Notion and Quora, Anthropic opened its Claude assistant more widely through a chat interface and API.
Anthropic, Product
- May 29, 2023
Researchers showed models can be aligned to human preferences with a simple classification-style loss, skipping the separate reward model and RL loop that RLHF requires.
Stanford University, Research
- Jul 18, 2023
Meta, with Microsoft as a launch partner, made Llama 2 weights free for research and commercial use, giving companies a capable model they could self-host.
Meta, Open weights
- Sep 27, 2023
A 7.3B-parameter Apache 2.0 model that Mistral reported beat Llama 2 13B on all its benchmarks, showing small open models could outperform larger ones.
Mistral AI, Open weights
- Nov 1, 2023
At the first AI Safety Summit, 28 countries and the EU, including the US and China, agreed that frontier AI risks require international cooperation.
AI Safety Summit (UK), Policy
- Dec 6, 2023
Google's first Gemini models came in Ultra, Pro and Nano sizes and were built from the start to handle text, images, audio, code and video.
Google DeepMind, Model release
- Dec 11, 2023
A sparse mixture-of-experts model using about 12.9B of its 46.7B parameters per token, bringing MoE efficiency to an openly licensed Apache 2.0 model.
Mistral AI, Open weights
- Dec 12, 2023
Microsoft's 2.7B-parameter model matched or beat models up to 25 times its size on complex benchmarks, strengthening the case for small language models.
Microsoft, Model release
2024
- Feb 15, 2024
Gemini 1.5 Pro neared Gemini 1.0 Ultra's performance and let selected developers test a 1M-token context window, about eight times its standard 128K limit.
Google DeepMind, Model release
- Mar 4, 2024
Anthropic introduced Haiku, Sonnet and Opus as a three-tier family with 200K-token context, giving developers a clear speed, cost and capability trade-off.
Anthropic, Model release
- Mar 17, 2024
xAI published Grok-1's raw 314B-parameter mixture-of-experts base checkpoint under Apache 2.0, giving researchers a very large, permissively licensed model to study and adapt.
xAI, Open weights
- May 13, 2024
OpenAI's omni model took any mix of text, audio, image and video input and could reply to speech in about the time a person takes in conversation.
OpenAI, Model release
- Jul 23, 2024
Meta's 405B-parameter model with 128K context was the largest Llama yet, and Meta presented it as the first openly available model on par with leading frontier systems.
Meta, Open weights
- Aug 1, 2024
The EU's risk-based AI law took effect, with bans, general-purpose model duties and high-risk requirements set to phase in over the following years.
European Union, Policy
- Sep 12, 2024
The first of OpenAI's o-series reasoning models spent extra time thinking before answering, showing that inference-time reasoning could lift math, science and coding results.
OpenAI, Model release
- Oct 22, 2024
An upgraded Claude 3.5 Sonnet could operate a computer by viewing screenshots, moving the cursor, clicking and typing, an early public step toward computer-using agents.
Anthropic, Product
- Nov 25, 2024
Anthropic open-sourced a protocol for linking AI assistants to tools and data, letting developers build a connector once and reuse it across compatible apps.
Anthropic, Product
- Dec 26, 2024
DeepSeek's 671B-parameter MoE model (37B active) reported results comparable to leading closed models from a full training run of 2.788M H800 GPU hours.
DeepSeek, Open weights
2025
- Jan 20, 2025
DeepSeek's MIT-licensed model, trained largely with reinforcement learning, reported reasoning performance on par with OpenAI o1 and arrived with six smaller distilled versions.
DeepSeek, Open weights
- Feb 2, 2025
Prohibitions on unacceptable-risk practices such as social scoring began applying, along with AI literacy duties, the Act's first binding obligations.
European Union, Policy
- Feb 24, 2025
Anthropic's first hybrid reasoning model let users switch on extended thinking, and debuted alongside Claude Code, a terminal tool for delegating coding tasks.
Anthropic, Model release
- Mar 25, 2025
Google's reasoning-focused Pro model debuted at the top of the LMArena leaderboard and led several math, science and coding benchmarks.
Google DeepMind, Model release
- Apr 5, 2025
Meta's first mixture-of-experts Llamas were natively multimodal, with Scout advertising a 10M-token context window; the larger Behemoth was previewed but not released.
Meta, Open weights
- Apr 29, 2025
Alibaba released eight Apache 2.0 models, from 0.6B dense to a 235B mixture-of-experts flagship, each able to switch between step-by-step thinking and fast replies.
Alibaba (Qwen), Open weights
- May 22, 2025
Anthropic pitched Opus 4 as the world's best coding model, citing 72.5% on SWE-bench, and made Claude Code generally available with beta IDE extensions.
Anthropic, Model release
- Jul 2025
Moonshot AI released a 1-trillion-parameter mixture-of-experts model with 32B active parameters under a modified MIT license, aimed at agentic tasks such as tool use.
Moonshot AI, Open weights
- Jul 9, 2025
Trained with large-scale reinforcement learning on xAI's Colossus cluster, Grok 4 shipped with native tool use and a multi-agent Heavy tier aimed at hard reasoning tasks.
xAI, Model release
- Aug 2, 2025
Obligations for providers of general-purpose AI models, including LLMs, began applying, with stricter duties for the most capable models deemed to pose systemic risk.
European Union, Policy
- Aug 5, 2025
OpenAI's first open-weight language models since GPT-2: Apache 2.0 reasoning models sized to run on a single 80 GB GPU (120b) or within 16 GB of memory (20b).
OpenAI, Open weights
- Aug 7, 2025
GPT-5 combined a fast model and a deeper reasoning model behind an automatic router and reached ChatGPT's free tier, putting reasoning in front of mainstream users.
OpenAI, Model release
- Nov 18, 2025
Google's new flagship topped LMArena with a 1501 Elo score and launched alongside Antigravity, an agentic development platform, across Search, the Gemini app and developer tools.
Google DeepMind, Model release
- Dec 2, 2025
Mistral Large 3, a 675B-parameter MoE with 41B active, and three small Ministral 3 models arrived under Apache 2.0, all with image understanding.
Mistral AI, Open weights
2026
- Feb 5, 2026
Anthropic's first Opus-class model with a 1M-token context window (in beta) added 128K-token outputs and agent teams in Claude Code for longer autonomous work.
Anthropic, Model release
- Mar 5, 2026
GPT-5.4 merged GPT-5.3-Codex's coding strengths with built-in computer use, scoring 75.0% on OSWorld-Verified against a human baseline of 72.4% cited by OpenAI.
OpenAI, Model release
- Apr 2, 2026
Google DeepMind's open models, from phone-scale E2B to a 31B dense model and a 26B mixture-of-experts, shipped under Apache 2.0 rather than earlier Gemma's custom terms.
Google DeepMind, Open weights
- Apr 7, 2026
Anthropic withheld Claude Mythos Preview from general release over its skill at finding and exploiting software flaws, limiting it to defensive partners such as AWS, Apple and Microsoft.
Anthropic, Model release
- Apr 23, 2026
OpenAI positioned GPT-5.5 for long, multi-step computer work, reporting leading scores on agentic benchmarks such as Terminal-Bench 2.0 (82.7%) and OSWorld-Verified (78.7%).
OpenAI, Model release
- Apr 24, 2026
DeepSeek open-sourced a 1.6-trillion-parameter Pro model and a 284B Flash model, both with 1M-token context, making million-token windows the default across its services.
DeepSeek, Open weights
- May 19, 2026
Unveiled at Google I/O, Gemini 3.5 Flash opened Google's 3.5 generation and was generally available in the Gemini API on day one; a larger 3.5 Pro was promised later.
Google DeepMind, Model release
- Jun 9, 2026
Anthropic made a Mythos-class model generally available as Fable 5, routing some cyber, bio and distillation queries to Opus 4.8, while the less-restricted Mythos 5 went to vetted partners.
Anthropic, Model release
- Jul 27, 2026
The Digital Omnibus amendments pushed high-risk AI obligations to December 2027 and August 2028, days before the Act's general application date of August 2, 2026.
European Union, Policy
- Aug 2026
Alibaba's Qwen3.8 open-weight releases included a 27B Apache 2.0 model and a 2.4-trillion-parameter mixture-of-experts flagship (95B active) under a separate custom license.
Alibaba (Qwen), Open weights
- Sep 3, 2026
OpenAI opened its GPT-6 generation with Astra, which it said met its 'Critical' cybersecurity threshold, so it shipped with safeguards that refuse advanced cyber tasks such as proof-of-concept exploits.
OpenAI, Model release
- Oct 7, 2026
OpenAI began rolling GPT-6 out across ChatGPT, including the free tier, and added Intelligent UI, which can answer with charts, forms and small interactive tools as well as text.
OpenAI, Product
Building with LLMs? Get a vendor-neutral plan.
Tell us what you're building. Within 2 business days you get a model shortlist, an architecture outline and a cost estimate, plus optional introductions to vetted implementation partners.
- Free and vendor-neutral: no model provider pays for placement.
- Your details go only to partners you agree to.