The Coming Intelligence Explosion
Explaining, for those out of the loop, what is coming and how we know
Industry leaders, independent experts, and top forecasters believe that the world is likely on track for an imminent intelligence explosion, where AI surpasses humans in performing cognitive labor. But why think this? And what might be the implications?
AI progress has been rapid across many different capabilities benchmarks. METR found that the length of tasks AIs can complete with 50% accuracy has been doubling every seven months. In more recent months, doubling times have been shorter than seven months—the most recent doubling took just four months.
Growth in AI cognitive labor has been hundreds of times faster than growth in human cognitive labor. As early as 2025, MacAskill and Moorhouse noted, “even if current rates of AI progress slow by a factor of 100x compared to current trends, total cognitive research labour (the combined efforts from humans and AI) will still grow far more rapidly than before.” In other words, even if progress slows considerably, AI will still massively accelerate global research progress. And since then, progress has only sped up.
Current capabilities are already hugely impressive and growing more so by the day. Anthropic refused to release its newest model to the general public, for fear that it would be used for cyber-attacks. In June 2024, AI models could only solve about 1% of FrontierMath problems—a set of 350 original math problems written by experts, so difficult that human specialists would struggle to solve them. Today, models can solve more than half. Models have been similarly impressive in coding.
Many of the drivers of AI capabilities growth have been multiplying many times each year.
The cost to run an LLM at any given level of capabilities has declined around 40x per year.
Total computing power has gone up 5x per year.
Efficiency of compute has gone up 3x per year.
Chip performance per dollar has gone up 37% per year.
If these trends continue for six years, it would be equivalent to more than a millionfold increase in effective compute (with compute increasing and getting more efficient). That will take place alongside other capabilities improvements. Progress can likely continue through 2030, making it possible to train a model that eclipses GPT-4 in compute by as much as GPT-4 (released in 2023) eclipses GPT-2 (released in 2019).
The best models in 2019 were basically useless. By 2023, they had more general knowledge than any person, and now they routinely surpass Ph.D.-level experts on hard questions. If we get just one more jump of this size in the next ten years, AI capabilities will far surpass the best humans across many domains.
In short, then, AI is able to perform longer tasks with higher accuracy, more cheaply. Plausible extrapolations of these trends have AIs soon surpassing humans in cognitive domains. Crucially, one of the domains in which they’ve been most useful is AI research—the better they are, the faster the rate of improvement will be.
This is likely to majorly shake up the world. If cognitive labor multiplies many times each year, it grows to ridiculous degrees over many years. Even a mere growth of 3x per year—far below current trends—is a nearly 60,000-fold increase in a decade. Once AI reaches human capabilities, it will be as if the population of human researchers more than doubled annually, while also becoming smarter and more capable.
Cognitive labor drives growth, because economic growth is hugely bottlenecked by the production of new ideas. My colleagues at Forethought calculated that even by pretty conservative estimates, this could drive a hundred years of economic growth in just ten. Just imagine compressing all growth from the last hundred years into the span of a decade.
This kind of rapid technological capabilities advancement is likely to lead to advances in robotics and engineering, enabling an industrial explosion. New robotic capabilities will help convert cognitive labor into physical production. This could facilitate similar feedback loops: robot factories could build new and better robot factories, which build more and better robot factories, and so on.
Thus, it is reasonably likely that AI will soon surpass humans in performing cognitive labor, and this will cause extremely rapid economic growth. The exact effects of this are hard to forecast, but could be quite dramatic—a new industrial revolution, more significant than the first.
Well said.
And, of course, there is a subtext: humanity is about to hand the world to the machines, without knowing how to robustly align them with our vision of the good.
While I believe we will see an intelligence explosion at some point soon, a few reasons current LLM trends probably won't continue at this pace: most SOTA AI systems got there by learning from human data and knowledge. It's a lot easier to catch up to the frontier of knowledge than it it is to surpass it by making new discoveries. LLMs can do this, but notice they have only done it in math and CS where you easily verify the solution quickly. It's harder to discover a new effective drug this way, because your data center can't yet simulate 1000 different human bodies taking it. Things like AlphaFold can help of course, so we can expect to see things like AlphaFold for all sorts of domains.
And there are tons of stones left unturned with modern methods. Everyone is focused on LLMs right now, but neural networks have shown their power in many more domains. LLMs are letter predictors which had a crazy ability To understand the training data, and People are now working on things like video prediction.