Have you noticed that when you open a new chat with ChatGPT, it has no memory of your previous chats? Or that your self-driving car keeps making the same mistake every time it passes through the tunnel?
That’s because modern AI systems do not yet learn continuously as they go. Retraining only occurs manually with human oversight; engineers collect and clean incoming data, retrain the system, and meticulously monitor its performance before sending it back into the world.
Modern artificial neural networks suffer from what is known as the “problem of catastrophic forgetting”: when you teach them new things, they tend to forget old things. Other limitations include lack of common sense and fine motor skills.
Billions of dollars are being spent on trying to solve these challenges. But we are late to the game. Nature discovered solutions to these problems over 100 million years ago in the brains of the first mammals. All modern mammals solve these problems effortlessly—even a small rat.
A rat acquires new information without forgetting old information, exhibits exquisite common sense, has fine motor skills that surpass even the most sophisticated robotic arms, and can plan its routes through a complex maze better than any modern robot.
How do rats do it? In your brain (just as in all mammal brains) there are two systems of thinking; one in which you pause to perform some mental operations, and the other in which you automatically make choices. This duality shows up in AI research, psychology, and neuroscience: in psychology these are called “System 2” versus “System 1” (after Daniel Kahneman’s famous book Thinking Fast and Slow); in neuroscience they are called “goal-directed decision making” and “habitual decision making;” and in AI research they are called “model-based” and “model-free.”
One of the crucial things missing in modern AI systems is this slower version of thinking. This inner “world model” is the basis of our imagination—what enables us to close our eyes and plan how we want to get to work, or what we want to say in a speech, or how to place our fingers on our guitar to play a specific chord. It is what gives us common sense and enables us to incorporate old information without disrupting new information.
Some AI systems can simulate possible futures—Google maps can chart a path and AlphaZero can play out possible future moves when playing chess. But AlphaZero and other AI systems still can’t yet engage in reliable planning in real-world settings, outside of the simplified conditions of a board game or a map. In real-world settings, simulating possible plans requires dealing with imperfect “noisy” information, an infinite space of possible next actions, and ever-changing internal needs, all feats rats perform effortlessly.
Because of these limitations, the recent success of large language models has taken many AI researchers, cognitive scientists, and neuroscientists by surprise. It turns out that if you scale up a model-free, habitual, System 1 artificial brain with a lot more neurons and a lot more data, it starts being capable of many of the feats that many researchers thought would only be possible with a model-based, goal-oriented System 2 brain. GPT-4 answers commonsense questions surprisingly well despite the fact that it never pauses to render a simulation of the external world; indeed, it has never seen our world, it has only ever learned from words. GPT-4 can also explain its own reasoning with an eerie level of coherence, despite the fact that we know it did not pause to think about how it reasoned about a prior answer. GPT-4 is an incredible feat of “fast” thinking.
“The goal of AI is not to recreate the human brain, which has its own portfolio of flaws, but to transcend it.”
However, if we just continue to scale up these systems with more data and more neurons, they are likely to remain brittle, frozen in time, and risk making mistakes in unpredictable ways that we cannot explain. They may never acquire the fine motor skills we want them to have. Should they achieve human-level performance, it will suggest they do so while working in a very different way than our own brains, which means we will be rolling dice that they will not spontaneously start making mistakes in ways we did not anticipate.
The goal of AI is not to recreate the human brain, which has its own portfolio of flaws, but to transcend it. To take the good and re-engineer out the bad. But the current approach of ignoring the human brain entirely, of barreling forward with scaling up neural networks by giving them more neurons and more data, may risk missing a crucial aspect of human intelligence that we will want to see in our AI systems.
The human brain evolved over a long period of time through a long process of incrementally acquiring intellectual faculties, each stacked on top of another. Modern AI systems are missing past breakthroughs that occurred in brain evolution. If we slow down to make sure we add them, the AI systems we will end up creating will be safer, more robust, and better equipped to fulfill AI’s promise. Or at the very least, we will tip the odds in favor of a good outcome in this new, odd, scary, and possibly utopic world of AI we have now entered.
