←  Artificial Minds An Essay · October 2026
A pencil drawing. A curly-haired child reads at a small wooden desk, and roots spread from under the desk. The roots carry a monkey, a shrew-like animal, a lizard, a fern, a fish, a coiled shell and a single cell. Across an empty gap, a small robot reads at a matching desk beside two tall stacks of books, and pages fall from one stack into a frame of glass shelves and small blocks under the floor.

Who learns faster, a toddler or a chatbot?

It depends on the stage. Today’s chatbots need trillions of words to grow up, but they pick up a new skill in a minute. AI is slow to learn, quick to catch on.

A toddler at the breakfast table points at a cup and says “more milk.” Most children can put two words together like this by their second birthday, after hearing roughly nine million words.1,2 Nobody gave the toddler the internet to read.

A chatbot such as ChatGPT, Claude or Gemini runs on an AI model, and the model reads trillions of words before it can hold a conversation. Researchers call this the question of sample efficiency: how much a learner gets out of each example. On that measure the toddler wins by a mile.

The short answer is that the two learn in different ways, at different times. A child starts with a brain that evolution shaped for learning, and she learns from a world she can see and touch. A model starts from random numbers and learns from a huge pile of text. Once trained, though, the model is very quick at something a child finds hard.

Many companies build models, and the numbers for each model change every few months. To keep things simple, this essay follows one model: Llama 3.1, which Meta released in 2024.3 From here on it is “our model.” The other big models are trained in broadly similar ways.

The essay follows both learners through four stages, side by side.

Two learners, four stages

A HUMAN OUR MODEL pre-training pre-training a single cell a fish a reptile a mammal a primate a modern human random weights DNA a baby pages of text a child a reward released, weights fixed four things at once about a whole novel about eight novels sleep weights stay fixed 1 BEFORE BIRTH Who starts with more? a single cell 3.5 billion years ago a fish 500 million years ago a reptile 300 million years ago a mammal 200 million years ago a primate 60 million years ago a modern human 300,000 years ago random weights near a blank slate evolution’s head start travels in DNA: one copy holds about 3 billion letters pre-training no known AI 2 GROWING UP Who needs more examples? a baby learns in every waking moment, from sounds and faces pre-training continues trillions of words, about 4,000 lifetimes of reading a child about 45 million words heard by age ten post-training practice with rewards released weights fixed 3 IN A CONVERSATION Who picks up something new faster? working memory about four things at once working memory about a whole novel at once the newest chatbots: about eight novels learns straight away; the lesson lasts while it stays in view 4 THE REST OF LIFE Who remembers tomorrow? learns for life in sleep, the brain replays the day into lasting memory weights stay fixed a new version comes months later keeps growing version 1 version 2 A HUMAN OUR MODEL pre-training pre-training a single cell a fish a reptile a mammal a primate a modern human random weights DNA a baby pages of text a child a reward released, weights fixed four things at once about a whole novel about eight novels sleep weights stay fixed 1 BEFORE BIRTH Who starts with more? a single cell 3.5 billion years ago a fish 500 million years ago a reptile 300 million years ago a mammal 200 million years ago a primate 60 million years ago a modern human 300,000 years ago random weights near a blank slate evolution’s head start travels in DNA: one copy holds about 3 billion letters pre-training no known AI 2 GROWING UP Who needs more examples? a baby learns in every waking moment, from sounds and faces pre-training continues trillions of words, about 4,000 lifetimes of reading a child about 45 million words heard by age ten post-training practice with rewards released weights fixed 3 IN A CONVERSATION Who picks up something new faster? working memory about four things at once working memory about a whole novel at once the newest chatbots: about eight novels learns straight away; the lesson lasts while it stays in view 4 THE REST OF LIFE Who remembers tomorrow? learns for life in sleep, the brain replays the day into lasting memory weights stay fixed a new version comes months later keeps growing version 1 version 2
Figure 1.Both learners run down the figure: the child is ahead before birth, growing up and over a lifetime, and our model is faster inside a conversation. Time is not drawn to scale, and the dates and growth lines are approximate.2,3,4,5,6,7,8,9

1 · Before birth

Who starts with more?

A baby arrives with a brain built for learning. Our model starts out as random numbers.

A child

Born with a brain that evolution spent billions of years shaping. Within months of birth, a baby already expects a solid object to stay solid.

Our model

Starts as billions of random numbers. Its first lesson, pre-training, has to build everything from scratch.

Babies have a head start. Young babies already expect certain things about objects, places, numbers and people. A baby looks longer, as if surprised, for example, when one solid object seems to pass through another.10

Our model gets no such gift. Before training, its weights are random. In pre-training, the model reads an enormous amount of text and guesses each next word. After every guess the weights shift a little, so the next guess comes out better. Everything the model learns ends up stored in those numbers.

Evolution passes its gift on in DNA. One copy is about 3 billion letters long. Our model’s weights take about a thousand times as much space to store.3,5 DNA is far too small to hold a copy of everything a child will know. Something in how her brain grows seems to make her a fast learner.

DNA is small next to our model

Our model’s weights: about 810 GB One copy of a baby’s DNA: about 760 MB our model’s weights about 1,000 times as much to store one copy of a baby’s DNA about 3 billion letters
Figure 2.The area of each square shows how much it takes to store. Our model’s weights need about a thousand times the space of one copy of a baby’s DNA. Sizes are approximate: about 760 megabytes for the DNA and about 810 gigabytes for the weights.3,5
I think there’s something going on where pre-training is not like the process of humans learning, but it’s somewhere between the process of humans learning and the process of human evolution. We get many of our priors from evolution. Our brain isn’t just a blank slate.

Dario Amodei, head of Anthropic, interview, 13 February 2026.11

That’s why I call pre-training this crappy evolution. It’s the practically possible version with our technology and what we have available to us to get to a starting point where we can do things like reinforcement learning and so on.

Andrej Karpathy, AI researcher and a founding member of OpenAI, podcast interview, 17 October 2025.12

Ilya Sutskever doubts that evolution explains the whole gap. Evolution gave humans a lot for vision, hearing and movement, he says, “But in language and math and coding, probably not.” Children still learn these from far fewer examples than a model needs.13

Somehow a human being, after even 15 years with a tiny fraction of the pre-training data, they know much less. But whatever they do know, they know much more deeply somehow. Already at that age, you would not make mistakes that our AIs make.

Ilya Sutskever, co-founder and former chief scientist of OpenAI, interview, 25 November 2025.13

So whatThe child starts with a brain built for learning. Our model has to build its own way of learning out of data, and that takes a lot of it.

2 · Growing up

Who needs more examples?

A child hears millions of words. Our model reads trillions.

A child

Hears about 45 million words by age ten, and sees, touches and tries things all day long.

Our model

Reads about 11.7 trillion words, about 4,000 lifetimes of reading. Then it practises with rewards.

Side by side, the two piles of words are very different in size. A child’s words fit in a corner of one person’s lifetime of reading, and that lifetime fits in a corner of our model’s training text.2,3,4

A child hears millions of words, and our model reads trillions

AREA SHOWS THE NUMBER OF WORDS Our model’s training text: about 11.7 trillion words in the next figure, this red square is the small one in the corner our model’s training text about 11.7 trillion words, or about 4,000 lifetimes of reading One person’s lifetime of reading: about 3 billion words a person’s lifetime of reading One person’s lifetime of reading, drawn 50 times wider: about 3 billion words reading 8 hours a day from age 6 to 80 the same speck, drawn 50 times wider a person’s lifetime of reading about 3 billion words A child by age ten: about 45 million words A child by age two: about 9 million words a child by age ten about 45 million words a child by age two about 9 million words
Figure 3.One person’s whole lifetime of reading is the grey speck in the corner of our model’s training text. A ten-year-old’s words fit in a corner of that lifetime. Area shows the amount.2,3,4,14,15

Words are a small part of what a child takes in. Her eyes send a steady stream of data to her brain from the day she is born. By Yann LeCun’s way of counting, a four-year-old has seen about 30 times as much data as our model’s whole training text. A stricter count still gives about four times as much.3,16,17 Counted this way, the picture flips. Our model’s whole training text becomes the small square in the corner.

Our model’s whole training text is a corner of what a child sees

COUNTING WORDS as in the last figure COUNTING WHAT THE EYES TAKE IN Counting words: our model’s training text is the big square A person’s lifetime of reading our model a person Counting what the eyes take in: a child’s eyes fill the big square Our model’s training text is now the small square a child our model LeCun’s count: about a million gigabytes A stricter count: about 120,000 gigabytes Our model’s whole training text: about 30,000 gigabytes a four-year-old’s eyes 16,000 waking hours at about 20 megabytes a second LeCun’s count about a million gigabytes, about 30 times the text a stricter count about 120,000 gigabytes, about 4 times the text now our model’s whole training text is the small square in the corner about 30,000 gigabytes
Figure 4.In a count of what the eyes take in, our model’s whole training text is the small square in the corner. By Yann LeCun’s count, a four-year-old’s eyes have taken in about 30 times as much data, and by a stricter count about 4 times. Area shows the amount. Text is counted as LeCun counts it, at two bytes a token.3,16,17

A child’s data also comes in a useful order. She chooses what to look at, picks things up and tries again when something surprises her, so each example teaches her more. A handful of pineapples is enough for her to know the next one.18,19,20 Today’s big models learn from images, sound and video as well as text.21 Our model still gets no say in what it reads next.

A brain and a model are built the opposite way round. A brain has about 100 trillion connections, roughly 250 times as many as our model has weights, and a lifetime of a few billion seconds to fill those connections.3,22

Our brains are very different. We only live for about two billion seconds. We don’t have trillions of experiences, just a few billion.

Geoffrey Hinton, winner of the 2024 Nobel Prize in Physics for work on neural networks, interview, 30 October 2025.23

After pre-training, our model gets a second stage of schooling, called post-training. It practises answering questions, and good answers are rewarded. Then its builders fix the weights and release it.3

So whatOur model makes up for a weak way of learning with a huge amount of reading. A child learns more from each example, and she gets richer examples.

3 · In a conversation

Who picks up something new faster?

Here the model pulls ahead.

A child

Holds about four things in mind at once. Needs days to read a long book, and keeps the gist.

Our model

Holds about a whole novel in its working memory, and the newest chatbots hold about eight. Uses a new example straight away.

How much can a model keep in view at once? Our model holds close to 100,000 words, about one long novel. The newest chatbots hold about 750,000 words, about eight novels.3,7,8

A user can paste the first four Harry Potter books, close to half a million words, into one of the newest chatbots. A person reading eight hours a day would need about four days to get through them.4 The chatbot takes them all in at once. A minute or two later it can answer a question about any page, or write a new scene in the same style.

Four days of reading, taken in at once

Four thick books drawn in pencil, standing side by side. Their engraved covers show a castle under a winged ball, a snake coiled in front of a stone door, a stag under a full moon, and a flaming goblet. a person about four days of reading one of the newest chatbots a minute or two
Figure 5.A person needs about four days to read the first four Harry Potter books. One of the newest chatbots takes them all in at once.4,7,8

The same trick teaches a chatbot new skills. Given a team’s guide to writing reports, plus two good reports, a model can draft a third in the same style. Its weights stay exactly as they were. The new skill sits in the context. Researchers call this in-context learning.

The lesson lasts as long as it stays in the context.

If the guide drops out of view, because the chat grew too long or a new chat began, the model no longer has it. Tomorrow it will know the guide only if someone pastes it in again.

The weights stay locked, and the lesson lives in the context

The lesson lasts as long as it stays in view 1 The guide is in view style guide two good reports write the March report house style CONTEXT WINDOW in view weights locked 2 The guide scrolls out of view out of view style guide two good reports write the March report wrong style CONTEXT WINDOW in view weights unchanged 3 A new chat starts empty no guide in view write the April report ? generic style CONTEXT WINDOW in view weights unchanged The lesson lasts as long as it stays in view 1 The guide is in view 2 The guide scrolls out of view 3 A new chat starts empty nothing has scrolled away yet style guide two good reports write the March report house style style guide two good reports write the March report wrong style the old chat is closed no guide in view write the April report ? generic style out of view CONTEXT WINDOW in view weights locked
Figure 6.Our model writes in the team’s style for as long as the guide stays in its context window. Its weights stay locked in all three scenes. Illustrative.7,24

People work the other way round. We humans hold about four chunks in mind at once, such as a word or a familiar phrase.6 Within a minute of hearing a sentence, most people have lost its exact words and kept its meaning.25 What people do keep can last for years.

Working memory, person and machine

a person about four chunks at once each one a word or a familiar phrase A person: about four chunks milk eggs bread ice cream our model close to 100,000 words about one long novel Our model: close to 100,000 words, about one long novel the newest chatbots about 750,000 words about eight novels The newest chatbots: about 750,000 words, about eight novels the first four Harry Potter books about 460,000 words room to spare
Figure 7.A person holds about four things in mind at once. Our model holds about one novel, and the newest chatbots about eight. A chunk can be a whole phrase, so the units differ.3,6,7,8,26

The most striking test so far used a real language. Kalamang is spoken by fewer than 200 people in Indonesian Papua, and almost none of it is online. In 2024 Google gave one of its models a 573-page grammar of Kalamang, a word list and some example sentences, all in a single prompt. It translated English into Kalamang almost as well as a researcher who had studied the same book for weeks. It did worse translating the other way.21,27 Most of the help came from the book’s example sentences.28

How can a model learn without changing its weights? Pre-training reads so much text that the model ends up good at spotting patterns in whatever text it is shown. The slow learning of pre-training builds a fast learner.24

A slow loop builds a fast one

The outer loop and the inner loop OUTER LOOP: PRE-TRAINING reads text, guesses each next word, adjusts the weights a little after every guess the weights change slowly, over trillions of words INNER LOOP in-context learning style guide two good reports a third report in the same style the weights stay as they were
Figure 8.Pre-training, the slow outer loop, adjusts the weights a little after every guess. In the fast inner loop, our model drafts a third report in the same style as the two in view, and its weights stay as they were. Illustrative.24,29
But I do think that anything that’s in the weights, it’s a hazy recollection of what you read a year ago. Anything that you give it as a context at test time is directly in the working memory.

Andrej Karpathy, podcast interview, 17 October 2025.12

So whatInside one conversation, the model is the quicker learner by far. The catch comes when the conversation ends.

4 · The rest of life

Who remembers tomorrow?

What a child learns at school on Monday is still with her on Friday. Our model’s weights stay as they were.

A child

Keeps learning for life. Sleep helps turn the day into lasting memory.

Our model

Its weights stay fixed after release. Saved notes help, and a new version arrives months later.

People keep learning all their lives. A new employee picks up the job week by week and builds on what she learned the week before. The brain does this with two memory systems. One stores a new experience quickly, after a single exposure. The other learns slowly. During sleep the fast one replays the day to the slow one, which takes the lessons in a little at a time and keeps its older knowledge.9,30

Two memory systems in the brain, and a gap in the machine

A HUMAN Hippocampus fast: stores an experience after one exposure replay during sleep and rest Neocortex slow: folds it in and keeps its older knowledge AN ARTIFICIAL MIND The context fast: learns inside one conversation, lasts while in view “But where’s the hippocampus?” Karpathy, 2025 The weights slow: set by training, fixed between versions new training tends to overwrite old skills: catastrophic forgetting
Figure 9.An artificial mind has a fast memory in its context and a slow one in its weights. No general method yet copies lessons from the first into the second. Schematic.9,12,30,31,32

Networks have no general way yet to copy a lesson from the context into the weights. When a network is trained on new material, the new learning tends to wipe out older skills. Researchers call this catastrophic forgetting. A network trained on thousands of tasks in a row slowly loses much of its ability to learn, until it does no better than a far simpler network.32,33 That is one reason companies leave the weights alone after release. As of October 2026, none of the big AI companies had announced a general model that updates its own weights from its conversations.

The chatbot’s software can still help. When a user tells a chatbot “keep it short, please,” the software may save that as a note. Next time it pastes the note into the context, and the model reads it there.34,35 Memory features, summaries of long chats and searches of saved files all work this way. They put text back in view, and the weights stay as they were.

What “keeping notes” means

THE HARNESS Monday Keep it short, please. the context instructions notes from before: the chat so far: none yet “Keep it short, please.” THE MODEL weights fixed Got it. the model writes a note, and the harness saves it notes file User prefers short answers User prefers short answers Tuesday a new chat the context instructions notes from before: the chat so far: User prefers short answers “Any ideas for dinner?” the harness pastes the note in Any ideas for dinner? THE MODEL same weights as Monday Pasta with garlic and lemon. short, as asked on Monday The harness can also add documents and tool results.
Figure 10.The model’s weights stay fixed. Everything it seems to remember between chats is text the harness saves and puts back in view. Schematic.34,35

The weights change only when a company trains a new version, months later. Some companies let users choose whether their chats may help train it.36

All learning is continual. It’s not a special phase. We always act and we learn. And that’s just the normal way of thinking. I’m not weird. The field is weird. They feel they need to call it continual learning. It’s just learning.

Richard Sutton, who shared the 2024 Turing Award for work on learning by trial and reward, Training Data podcast, 18 August 2026.37,38

So whatA model that cannot learn after release has to learn almost everything before it. That is a big part of why it needs so much data.

What’s missing

What would close the gap?

Researchers point to four abilities that today’s models lack.

A model would need to keep learning after release, and to learn from the physical world. It would also need to choose its own experience, as a child does, and to reason about problems unlike anything in its training. Each card below lists researchers at universities and AI companies who work on one of these four abilities, with their findings or arguments.

Four things still missing

A row of calendar pages that ends in a padlock: learning stops after release

1Learning after release

Today’s models stop changing their weights once training ends.

  • Richard Sutton

    Richard Sutton and his team trained networks on 2,000 tasks in a row. On an early task the networks scored 89 percent. By the 2,000th task they scored 77 percent, about as well as a far simpler network. The authors call this losing the “ability to learn on new examples.”33

  • Raia Hadsell

    Raia Hadsell and colleagues found a partial cure for forgetting in 2017. Their method “remembers old tasks by selectively slowing down learning on the weights important for those tasks.”32

  • Chelsea Finn

    In 2017 Chelsea Finn and colleagues trained a model on many small tasks so that it can pick up a new one from a few examples. In their words, the method “trains the model to be easy to fine-tune.”39

  • Demis Hassabis

    Demis Hassabis wants a learning method that “actually changes the model over time.” In January 2026 he said, “that technique has not been cracked yet.”40

A robot arm that reaches for a block: learning by acting on things

2Learning from the world

“We cannot download the motor control signals of the robots from the internet,” Jim Fan said in 2024.41

  • Yann LeCun

    Yann LeCun says language models “lack a model of the world.” “This is why we don’t have a domestic robot that is as agile as a house cat, or a truly autonomous car.”42

  • Fei-Fei Li

    Fei-Fei Li calls language models “wordsmiths in the dark; eloquent but inexperienced, knowledgeable but ungrounded.”43

  • Sergey Levine

    Sergey Levine borrows Plato’s cave: “The cave is the Internet, and the light shines from human intelligence, casting shadows of real-world interactions on the cave wall for the LLM to observe.”44

  • Rodney Brooks

    Rodney Brooks points out that “human dexterity relies on a rich sense of touch,” and videos of hands carry none of it. “Collecting just visual data is not collecting the right data.”45

  • David Silver

    David Silver runs the company Ineffable Intelligence. It is “creating a superlearner that discovers all knowledge from its own experience.”46

One path that splits three ways: a learner picks its own path

3Exploring by choice

Young children pick much of their own experience. A model learns from whatever data it is given.

  • Alison Gopnik says children “are particularly adept at exploration.” She sees language models as “cultural technologies, like writing, pictures and print.”19

  • Linda Smith

    Linda Smith studies what babies see. “The data relevant to any one thing or event is sparse,” she says, and babies cope by steering their own input.18

  • Emmanuel Dupoux

    Emmanuel Dupoux, Yann LeCun and Jitendra Malik titled their 2026 paper “Why AI systems don’t learn and what to do about it.” Their design switches between “learning from observation” and “learning from active behavior,” as animals do.47

  • Jürgen Schmidhuber

    Jürgen Schmidhuber has built curious machines since 1990. Since 1991 his machines have earned a reward when their picture of the world improves, so random noise is “no fun at all.”48

  • Pierre-Yves Oudeyer

    Pierre-Yves Oudeyer gave a robot a similar reward in 2007. The robot picked tasks that were neither too easy nor too hard, and moved on once it had mastered them.49

A tight cluster of dots with one dot far outside it: a case far from the training data

4Reasoning far from the training data

In 2025 OpenAI’s o3 beat people’s accuracy on text puzzles built on abstract rules. When Melanie Mitchell’s group checked its rules, about 29 percent fit the examples for the wrong reasons. For people the figure was 2.7 percent.50

  • Ilya Sutskever

    Ilya Sutskever said in 2025: “these models somehow just generalize dramatically worse than people. It’s super obvious.”13

  • Melanie Mitchell

    Melanie Mitchell calls this “jagged intelligence.” “In general, today’s AI systems perform extremely well until, often unexpectedly, they don’t.”51

  • Subbarao Kambhampati

    Subbarao Kambhampati sees language models as “giant non-veridical memories,” meaning memories that can be wrong. Their answers “still need to be checked by external verifiers.”52

  • Yoshua Bengio

    In 2020 Yoshua Bengio and Anirudh Goyal saw “a large gap” between machines and people when a problem looks unlike the training data.53

  • François Chollet

    François Chollet warns that “unlimited priors or unlimited training data allow experimenters to ‘buy’ arbitrary levels of skills.”54

Figure 11.Each card names one ability today’s models lack, with some of the researchers working on it. The author chose the grouping. Demis Hassabis doubts one part of the second gap, the need to act in the world. Speaking about Google’s video model Veo 3 in July 2025, he said: “there was a lot of theories about you’d need embodied intelligence or robotics … so that you would understand things like intuitive physics. But it seems like you can understand it through passive observation, which is pretty surprising to me.”55 In April 2026 the company Generalist AI reported pre-training its robot network on more than half a million hours of data from devices worn by people.56

The answer

So why so much data?

Each learner is quick at a different time.

  1. Before birth. A child starts with a brain that evolution built for learning. A model starts from random numbers.
  2. Growing up. A child learns from a world she can see, touch and choose. A model makes up the difference with trillions of words.
  3. In a conversation. Here the model is faster. It can use a new example within seconds, while the example stays in view.
  4. The rest of life. A child keeps what she learns. A model’s weights stay fixed after release, so it has to learn almost everything up front.

The holy grail

Continual learning

A model could keep learning after release, from every conversation, the way a child keeps learning from every day. Researchers call this continual learning. A model like that would no longer have to learn almost everything up front, so it might need much less data. When will it arrive? The people building these models disagree.

Demis Hassabis
You want to do it more than just having your data in the context window. You want to have something a bit deeper than that, which is, as you say, actually changes the model over time. That’s what ideally you would have. And that technique has not been cracked yet.

Demis Hassabis, head of Google DeepMind, January 2026.40

There’s a good chance that in the next year or two, we also solve that.

Dario Amodei, head of Anthropic, February 2026.11

They don’t have continual learning. You can’t just tell them something and they’ll remember it. They’re cognitively lacking and it’s just not working. It will take about a decade to work through all of those issues.

Andrej Karpathy, AI researcher, October 2025.12

Back at the breakfast table, the two-year-old learns from every conversation and keeps what she learns. A chatbot that did the same would learn from each talk with us humans and keep the lesson for the next day. If Amodei is right, that chatbot arrives around the toddler’s third birthday. If Karpathy is right, it arrives around her eleventh.

Where to read more

  1. Michael Frank, “Bridging the data gap between children and large language models” (2023). In three pages, Frank compares the words children hear with the size of a model’s training set.
  2. Brenden Lake and colleagues, “Building Machines That Learn and Think Like People” (2017). Lake and colleagues list the abilities a machine would need for human-style learning.
  3. Andrej Karpathy, “Animals vs Ghosts,” post on X (2025). Karpathy explains in a short post why he sees today’s AI models as a new kind of thing.
  4. Dario Amodei, interview of 13 February 2026. Amodei compares each stage of AI training with a kind of human learning, and he guesses when continual learning will arrive.
  5. Richard Sutton and Khurram Javed on the Training Data podcast (2026). Sutton and Javed explain why AI models stop learning after training, and they talk about fixes.
  6. Anthropic, “Effective context engineering for AI agents” (2025). Engineers explain how the software around a model chooses what goes into its context.

Sources

  1. Centers for Disease Control and Prevention. Milestones by 2 years, read 2 October 2026. https://www.cdc.gov/act-early/milestones/2-years.html
  2. Gilkerson, J., and colleagues. “Mapping the early language environment using all-day recordings and automated analysis.” American Journal of Speech-Language Pathology, 2017. https://pmc.ncbi.nlm.nih.gov/articles/PMC6195063/
  3. Llama Team, Meta. “The Llama 3 Herd of Models.” arXiv 2407.21783, July 2024. https://arxiv.org/abs/2407.21783
  4. Brysbaert, M. “How many words do we read per minute? A review and meta-analysis of reading rate.” Journal of Memory and Language 109: 104047, December 2019. https://reader.ku.edu/sites/reader/files/2024-01/How%20many%20words%20do%20we%20read%20per%20minute%20(1).pdf
  5. Nurk, S., and colleagues. “The complete sequence of a human genome.” Science 376: 44 to 53, April 2022. https://doi.org/10.1126/science.abj6987
  6. Cowan, N. “The magical number 4 in short-term memory: A reconsideration of mental storage capacity.” Behavioral and Brain Sciences 24: 87 to 114, 2001. https://doi.org/10.1017/S0140525X01003922
  7. Anthropic. “Context windows.” Claude developer documentation, read 2 October 2026. https://platform.claude.com/docs/en/build-with-claude/context-windows
  8. Google. “Long context.” Gemini API documentation, updated 22 June 2026. https://ai.google.dev/gemini-api/docs/long-context
  9. Diekelmann, S., and Born, J. “The memory function of sleep.” Nature Reviews Neuroscience 11: 114 to 126, 2010. https://pubmed.ncbi.nlm.nih.gov/20046194/
  10. Spelke, E. S. What Babies Know: Core Knowledge and Composition, Volume 1. Oxford University Press, 2022. Prologue, as posted by her laboratory. https://sites.harvard.edu/harvardlds/files/2022/08/Introduction-Prologue.pdf
  11. Amodei, D. Interview, 13 February 2026; quoted passages at about 00:09:23 and in the chapter beginning 00:29:42. Official transcript: interview page.
  12. Karpathy, A. Podcast interview, 17 October 2025; quoted passages at about 00:01:02, 00:11:17, 00:17:46 and 00:19:20. Official transcript: interview page.
  13. Sutskever, I. Interview, 25 November 2025; quoted passages at 00:10:40 and 00:24:35. Official transcript: interview page.
  14. Frank, M. C. “Bridging the data gap between children and large language models.” Trends in Cognitive Sciences 27(11): 990 to 992, November 2023. https://doi.org/10.1016/j.tics.2023.08.007
  15. OpenAI Help Center. “What are tokens and how to count them?” https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them
  16. Lex Fridman Podcast #416, “Yann Lecun: Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI,” March 2024. Official transcript; paraphrased passage at 00:05:12. LeCun’s figures: about 16,000 waking hours by age four and about 20 megabytes a second through the optic nerve, about 1015 bytes in all, against about 2 × 1013 bytes of text in a large training set, so about 50 times as much. Against our model’s 15.6 trillion tokens, counted as LeCun counts at two bytes a token (about 3.1 × 1013 bytes), the total is about 30 times as much. https://lexfridman.com/yann-lecun-3-transcript/
  17. Koch, K., and colleagues. “How much the eye tells the brain.” Current Biology 16(14): 1428 to 1434, July 2006. The measurements were made in guinea pigs. Scaled up to the million or so output cells of a human eye, the figures come to about one megabyte a second for each eye. At that rate a four-year-old’s visual total comes out about a ninth of LeCun’s, still about four times our model’s text. https://doi.org/10.1016/j.cub.2006.05.056
  18. Smith, L. B. “Coherence statistics, self-generated experience and why young humans are much smarter than current AI.” NeurIPS invited talk, 12 December 2023, abstract. https://neurips.cc/virtual/2023/eventlistwithbios/invited%20talk
  19. Gopnik, A. “The Golem vs. Stone Soup: Understanding How Children Learn Can Help Us Understand And Improve AI.” NeurIPS invited talk, 10 December 2024, abstract. https://neurips.cc/virtual/2024/eventlistwithbios/invited%20talk
  20. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. “Building Machines That Learn and Think Like People.” Behavioral and Brain Sciences 40, 2017. https://arxiv.org/abs/1604.00289
  21. Gemini Team, Google. “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.” arXiv 2403.05530, March 2024, version of 16 December 2024. Kalamang scores from Tables 4, 5, 55 and 56. The report’s text gives the human’s English-to-Kalamang score as 5.58, and its tables give 5.60. In the first version, of March 2024, the February 2024 Gemini 1.5 Pro scored 5.52 into Kalamang and 4.36 into English. https://arxiv.org/abs/2403.05530
  22. Heaven, W. D. “Geoffrey Hinton tells us why he’s now scared of the tech he helped build.” MIT Technology Review, 2 May 2023. https://www.technologyreview.com/2023/05/02/1072528/geoffrey-hinton-google-why-scared-ai/
  23. Persuasion. “Geoffrey Hinton on Artificial Intelligence,” The Good Fight with Yascha Mounk, 30 October 2025. https://www.persuasion.community/p/geoffrey-hinton
  24. Brown, T. B., and 30 co-authors, Dario Amodei last. “Language Models are Few-Shot Learners.” arXiv 2005.14165, May 2020. https://arxiv.org/abs/2005.14165
  25. Sachs, J. S. “Recognition memory for syntactic and semantic aspects of connected discourse.” Perception & Psychophysics 2: 437 to 442, 1967. https://link.springer.com/article/10.3758/BF03208784
  26. OpenAI. Models page, API documentation, read 2 October 2026. https://developers.openai.com/api/docs/models
  27. Tanzer, G., Suzgun, M., Visser, E., Jurafsky, D., and Melas-Kyriazi, L. “A Benchmark for Learning to Translate a New Language from One Grammar Book.” arXiv 2309.16575, version of 9 February 2024. https://arxiv.org/abs/2309.16575
  28. Aycock, S., Stap, D., Wu, D., Monz, C., and Sima’an, K. “Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?” ICLR 2025. arXiv 2409.19151. https://arxiv.org/abs/2409.19151
  29. Wang, J. X., and colleagues. “Prefrontal cortex as a meta-reinforcement learning system.” Nature Neuroscience, 14 May 2018, https://www.nature.com/articles/s41593-018-0147-8; and the Google DeepMind blog post of the same title by Wang, Kurth-Nelson and Botvinick, 14 May 2018, https://deepmind.google/discover/blog/prefrontal-cortex-as-a-meta-reinforcement-learning-system/
  30. McClelland, J. L., McNaughton, B. L., and O’Reilly, R. C. “Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.” Psychological Review 102(3): 419 to 457, July 1995. https://pubmed.ncbi.nlm.nih.gov/7624455/
  31. Kumaran, D., Hassabis, D., and McClelland, J. L. “What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated.” Trends in Cognitive Sciences 20(7), July 2016. https://stanford.edu/~jlmcc/papers/KumaranHassabisMcC16CLSUpdate.pdf
  32. Kirkpatrick, J., and colleagues, with R. Hadsell. “Overcoming catastrophic forgetting in neural networks.” Proceedings of the National Academy of Sciences, 2017. arXiv 1612.00796. https://arxiv.org/abs/1612.00796
  33. Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R., and Sutton, R. S. “Loss of plasticity in deep continual learning.” Nature 632: 768 to 774, 21 August 2024. https://www.nature.com/articles/s41586-024-07711-7 The quoted wording and the 89 and 77 percent figures are from the preprint, arXiv 2306.13812.
  34. OpenAI. “Memory and new controls for ChatGPT,” first posted 13 February 2024, with later updates. https://openai.com/index/memory-and-new-controls-for-chatgpt/
  35. Anthropic Applied AI team. “Effective context engineering for AI agents,” 29 September 2025. https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
  36. Anthropic. “Updates to Consumer Terms and Privacy Policy,” 28 August 2025. https://www.anthropic.com/news/updates-to-our-consumer-terms
  37. ACM. Press release on the 2024 A.M. Turing Award, 5 March 2025. https://acm.org/binaries/content/assets/press-releases/2025/march/turing-award-2024.pdf
  38. Sequoia Capital, Training Data, episode 98. “Rich Sutton and Khurram Javed: Why AI Models Stop Learning, and How to Start It Again,” released 18 August 2026. Official transcript. https://sequoiacap.com/podcast/rich-sutton-and-khurram-javed-why-ai-models-stop-learning-and-how-to-start-it-again/
  39. Finn, C., Abbeel, P., and Levine, S. “Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks.” ICML 2017; arXiv 1703.03400, March 2017. https://arxiv.org/abs/1703.03400
  40. Kantrowitz, A. Big Technology, Q&A with Demis Hassabis, 29 January 2026. https://www.bigtechnology.com/p/google-deepmind-ceo-demis-hassabis-946
  41. Sequoia Capital, Training Data. “Jim Fan on Nvidia’s Embodied AI Lab and Jensen Huang’s Prediction that All Robots will be Autonomous,” 2024. Official transcript. https://sequoiacap.com/podcast/training-data-jim-fan/
  42. Chen, C. “Yann LeCun’s new venture is a contrarian bet against large language models.” MIT Technology Review, 22 January 2026. Edited interview. https://www.technologyreview.com/2026/01/22/1131661/yann-lecuns-new-venture-ami-labs/
  43. Li, F.-F. “From Words to Worlds: Spatial Intelligence is AI’s Next Frontier,” 10 November 2025. https://drfeifei.substack.com/p/from-words-to-worlds-spatial-intelligence
  44. Levine, S. “Language Models in Plato’s Cave.” Learning and Control, 8 June 2025. https://sergeylevine.substack.com/p/language-models-in-platos-cave
  45. Brooks, R. “Why Today’s Humanoids Won’t Learn Dexterity,” 26 September 2025. https://rodneybrooks.com/why-todays-humanoids-wont-learn-dexterity/
  46. Ineffable Intelligence. “Mission,” company home page, read October 2026; and Silver, D. “A Note from Dave.” Ineffable Intelligence, 15 January 2026. https://www.ineffable.ai/
  47. Dupoux, E., LeCun, Y., and Malik, J. “Why AI systems don’t learn and what to do about it: Lessons on autonomous learning from cognitive science.” arXiv 2603.15381, 16 March 2026. https://arxiv.org/abs/2603.15381
  48. Schmidhuber, J. “Artificial Curiosity Since 1990,” web page, 2021, updated 2022 and 2023. https://people.idsia.ch/~juergen/artificial-curiosity-since-1990.html
  49. Oudeyer, P.-Y., Kaplan, F., and Hafner, V. V. “Intrinsic Motivation Systems for Autonomous Mental Development.” IEEE Transactions on Evolutionary Computation 11(2), April 2007. https://adapt.informatik.hu-berlin.de/pub/papers/ims.pdf
  50. Beger, C., and colleagues, including M. Mitchell. “Do AI Models Perform Human-like Abstract Reasoning Across Modalities?” arXiv 2510.02125, October 2025, version 4 of February 2026. https://arxiv.org/html/2510.02125v4
  51. Mitchell, M. “Jagged Intelligence: The Dangerous Unknowns at the Heart of LLMs.” The Yale Review, 8 June 2026. https://yalereview.org/article/melanie-mitchell-jagged-intelligence
  52. Kambhampati, S. “Can Large Language Models Reason and Plan?” arXiv 2403.04121, March 2024. https://arxiv.org/abs/2403.04121
  53. Goyal, A., and Bengio, Y. “Inductive Biases for Deep Learning of Higher-Level Cognition.” arXiv 2011.15091, November 2020. https://arxiv.org/abs/2011.15091
  54. Chollet, F. “On the Measure of Intelligence.” arXiv 1911.01547, November 2019; definition in section II.2.1. https://arxiv.org/abs/1911.01547
  55. Lex Fridman Podcast #475, “Demis Hassabis: Future of AI, Simulating Reality, Physics and Video Games,” 23 July 2025. Official transcript, at 00:18:06. https://lexfridman.com/demis-hassabis-2-transcript/
  56. Generalist AI. “GEN-1 / Scaling Embodied Foundation Models to Mastery,” blog post, 2 April 2026. https://generalistai.com/blog/gen-1