Jeff Dean on building useful AI systems

Jeff Dean is one of the GOATs of modern computing: he helped Google scale, co-founded Google Brain, and is now Google’s Chief Scientist. At Y Combinator’s Startup School 2026, YC Managing Partner Diana Hu interviews him about agents working for weeks, the chips, memory, context and evaluation they need, and which assumptions and unsolved problems founders should chase.

Agents may soon automate experiments across technical fields.

• Coding agents broadly match his qualified junior-engineer prediction.
• Improvements now extend beyond coding.
• Automated loops can split problems, test solutions, and combine results.

Specialized inference chips could make agents faster and cheaper.

• Hu compares the opportunity to Google’s faster in-memory search.
• Low-energy inference should broaden access.
• Specialized chips can beat general hardware on defined workloads.
• Fifty-fold lower latency could enable new products.

Some capable agents now work for days or weeks.

• Results depend on the model and problem domain.
• Software can be rebuilt in another language.
• Replacements can target better performance or safety.

Speech-recognition demand made CPUs unaffordable, prompting the TPU.

• Better speech models halved errors but required much more computation.
• The first TPU cut energy 30–80 times and latency 20–30 times.
• Flexible linear algebra supported changing algorithms.
• Dean urges founders to rethink bottlenecks from first principles.

Data movement can cost far more energy than calculation.

• Designers must track memory, chip-link, and network bandwidth.
• Batching saves energy but increases response time.
• Inference chips can reduce movement and precision.

Compression depends on understanding what can be restored.

• Lossy compression removes details that may need reconstruction.
• Deeper understanding should improve compression.
• Transformers have worked well so far.

Useful systems combine models, context, memory, tools, and evaluation.

• Direct context can be clearer than training knowledge.
• Complex work requires tool choice, sequencing, and evaluation.
• Failures reveal which instructions or skills are missing.
• Dean encoded code optimization into a benchmark-driven skill.
• Performance Hints also improved models’ reasoning about slow code.

Skills and competing attempts help agents outside familiar territory.

• Performance can collapse outside training.
• Guidance keeps agents near reliable operations.
• Evaluators can select promising attempts and discard failures.
• Google supplies internal agents with proprietary-tool skills.

Startups can compete where general models fail.

• Dean favors useful problems with 0% or 1% model success.
• Private data or specialist training can create advantages.
• Founders must judge how long those advantages will last.
• Specifications and tests make agent objectives concrete.

People must still choose worthwhile goals and question assumptions carefully.

• Strong tools add little to an unimportant problem.
• Written predictions help test judgment.
• Contrarian experiments sometimes expose better designs.
• Dean’s unreliable-chip example replaces perfect parts with redundancy.

TPUs and MapReduce emerged from reconsidering familiar designs.

• Google specialized hardware before machine learning looked important.
• MapReduce separated computation from parallelization and failure handling.
• Its abstraction kept large jobs reliable across computer failures.

Fast evaluators multiply possible experiments.

• Orchestration can divide goals and recombine experimental results.
• A learned chemistry simulator ran 300,000 times faster with similar accuracy.
• Faster validation could accelerate science, engineering, and AI.

A rejected model-distillation paper still became useful across industry.

• A reviewer predicted little impact.
• Operating large AI services made its value clearer.
• Google now uses distillation for efficient Gemini Flash models.

Meaningful work and enjoyable teammates guide his advice.

• Established organizations offer structure, expertise, and reach.
• Startups offer autonomy and reward with greater risk.
• Strong teams combine complementary skills, low egos, and learning.
• Teammates expand each person’s technical tool belt.

Open opportunities span hardware, learning, and human interaction.

• Efficient inference hardware remains an open opportunity.
• Current models consume far more data than humans.
• Continual learning and multi-agent interaction remain promising.
• Better tools could support civil conversation and human connections.