Grant Sanderson on the Locked-Box Thought Experiment for AI Writing
- Frontier Models And Capabilities
- Agents
- Jobs, GDP, And Economic Growth
- AI For Science
Watch the deep dive
Autoregression is actually a really weird way to produce stuff, if you think about it.
A thought experiment:
Imagine you have to write an essay under the following constraints.
You're locked in a box. Into that box, someone passes you a slip of paper with the essay so far. You're asked to provide the single next word. After providing that word, your memory is wiped.
You're then passed a new slip of paper with the current essay, including your most recently provided word. This repeats for thousands of turns until the essay is complete.
You cannot keep a separate plan, choose the points you wish to make, or circle back to the beginning. Just one blind word choice after the next, based on the existing string of words.
The essay may read somewhat coherently and be grammatically correct. But it would nonetheless be a very shitty essay, lacking insight, surprise, and interesting nonlinear connections.
Now imagine everyone judging your ability to write based on this final essay.
In his conversation with Dwarkesh Patel, Grant Sanderson of 3Blue1Brown uses this thought experiment to describe autoregressive generation. His point: it's no surprise that large language models (LLMs), under the current training paradigm, are bad at writing.
Section 1
Section 01- A thought experiment:
Section 2
Section 02- Imagine you have to write an essay under the following constraints.
Section 3
Section 03- You're locked in a box. Into that box, someone passes you a slip of paper with the essay so far. You're asked to provide the single next word. After providing that word, your memory is wiped.
Section 4
Section 04- You're then passed a new slip of paper with the current essay, including your most recently provided word. This repeats for thousands of turns until the essay is complete.
Section 5
Section 05- You cannot keep a separate plan, choose the points you wish to make, or circle back to the beginning. Just one blind word choice after the next, based on the existing string of words.
Section 6
Section 06- The essay may read somewhat coherently and be grammatically correct. But it would nonetheless be a very shitty essay, lacking insight, surprise, and interesting nonlinear connections.
Section 7
Section 07- Now imagine everyone judging your ability to write based on this final essay.
Section 8
Section 08- In his conversation with Dwarkesh Patel, Grant Sanderson of 3Blue1Brown uses this thought experiment to describe autoregressive generation. His point: it's no surprise that large language models (LLMs), under the current training paradigm, are bad at writing.
Tags
- Labor Automation
- Frontier Models
- Agent Orchestration
- Research Labor Productivity