Your senses dump roughly a billion bits per second into your skull. Your conscious mind can handle about ten. That gap — eight orders of magnitude, the largest unexplained number in brain science according to Zheng and Meister's 2024 Neuron paper — is the cleanest picture I know of what the industry is building toward: a periphery that filters, and an architect that reviews the delta.
I have spent the last year watching people argue this intuition as if it were settled theology. One camp says the future is a tree of specialists under a thin orchestrator. The other says scale and end-to-end learning will eat every hand-built hierarchy the way they ate chess engines and speech pipelines. Both sides have real receipts. This post is my attempt to sort which receipts are load- bearing, which are just good rhetoric, and where the tree intuition breaks.
Established fact, contested claim, and opinion are labeled as I go. Every figure has a named source. If a senior reader finishes this and says “you oversold the brain,” I have failed the assignment I set myself.
TL;DR
- The thesis is directionally right, for partly the wrong reasons. Biology, frontier MoE models, production routers, and the agentic-systems literature all converge on hierarchy — but the evidence that matters is architectural and economic (sparse activation, verification cheaper than generation, routing economics). The “brain distributes intelligence to the periphery” line is real neuroscience and a weak engineering argument.
- Your single strongest number is the compression ratio: ~10⁹ bits/s of sensory input versus ~10 bits/s of conscious throughput (Zheng & Meister, Neuron, 2024). That ~10⁸× gap maps cleanly onto MoE sparse activation (DeepSeek-V3 fires 37B of 671B parameters per token) and orchestrator-worker agents (Anthropic reports +90.2% quality at ~15× the tokens).
- Land on a conditional conclusion, not a manifesto. Trees of models win where tasks decompose cleanly and verification is cheap. Monoliths and end-to-end win where the problem is entangled and shared context is essential.
The billion-to-ten funnel
Start with the number, not the metaphor. Jieyu Zheng and Markus Meister, writing in Neuron on December 17, 2024 (DOI: 10.1016/j.neuron.2024.11.008), put it bluntly: human sensory systems gather data at ~10⁹ bits/s, but “the information throughput of a human being is about 10 bits/s.” They call this the largest unexplained number in brain science. Their framing is an outer brain — high-dimensional, massively parallel, fast sensory/motor processing across millions of channels — and an inner brain operating on a dramatically reduced stream, filtered to the few bits that matter for behavior.
That is exactly the architect/periphery split. Their supporting detail is just as useful: the retina produces about a million output signals, each the result of a local computation on the visual image, and retinal circuits already compress the visual signal by at least a factor of ten before anything reaches cortex.
The canonical retinal bandwidth figure is older and still standing. Koch, McLean, Segev, Freed, Berry, Balasubramanian, and Sterling measured guinea-pig retina at ~875,000 bits/s across ~100,000 ganglion cells, then scaled to the human's ~1,000,000 ganglion cells to get ~10⁷ bits/s — “roughly the rate of an Ethernet connection” (Current Biology, 2006). Lead author Kristin Koch; senior author Peter Sterling. The retina is not a camera. Lettvin, Maturana, McCulloch, and Pitts showed that in 1959 with frog “bug detectors.” Gollisch and Meister's 2010 Neuron review made the modern case. Mammalian retina communicates over at least thirty parallel channels — distinct retinal ganglion cell types — doing edge detection, direction selectivity, and motion detection before V1 (Krieger et al., 2017; Baden et al., Nature, 2016). Photoreceptors adapt locally across roughly ten orders of magnitude of light intensity via response compression, calcium-mediated cascade modulation, and pigment bleaching (Fain et al., Physiological Reviews, 2001). No cortex required.
Predictive coding is the cleanest mechanical analogy for “the architect reviews the delta.” Rao and Ballard (Nature Neuroscience, 1999): higher cortical levels send predictions down; lower levels send only the residual error up. Friston's free-energy principle generalizes that story. Local computation. Local plasticity. Only the surprise travels.
And here is where I have to put my own favorite slogan on a leash. The line that “the brain isn't impressive because it processes information centrally — it's impressive because it distributes intelligence to the periphery” is rhetorically delicious. Zheng and Meister do not endorse it. They present the 10-bit throughput as an unexplained paradox: why does the brain need billions of neurons to process 10 bits/s? Neuroscience has shown that distributed local preprocessing exists and is used heavily. It has not proven that distributed intelligence is the superior engineering strategy. Planes do not flap wings. Biology was forced into peripheral computation by a ~20W metabolic budget, wiring length, signal delay, a birth-canal ceiling on skull size, and a billion years of path dependence. Silicon has fast buses, no per-spike energy tax, and arbitrary reconfigurability. Use the brain as a hook and an intuition pump. Do not use it as a proof.
Two papers, one conflation
I had this half-remembered as “that paper by Meta's ex-AI lead about small specialists.” It is not one paper. It is two different arguments wearing the same trench coat in my memory, and a knowledgeable reader will smell the mix-up immediately.
Yann LeCun (ex-Chief AI Scientist, Meta) is not arguing for a swarm of tiny specialists. His case is that autoregressive LLMs are a dead end for human-level intelligence and that we need world models — JEPA / I-JEPA / V-JEPA — predicting in an abstract representation space rather than generating tokens. He announced his intention to leave Meta in November 2025, departed after twelve years, and launched Paris-based AMI Labs in early 2026 to build exactly that. His modularity story (configurator, actor, world-model, cost modules in the 2022 “A Path Towards Autonomous Machine Intelligence” paper) is about the internal modules of one agent, not a fleet of fine-tuned SLMs.
The data-bandwidth argument people actually remember from him is the rhetorical estimate that a four-year-old has seen ~50× more data than the biggest LLMs — roughly 10¹⁵ bytes of optic-nerve traffic versus ~2×10¹³ bytes of LLM training tokens. Treat that as LeCun's rhetorical estimate, not a settled result: his per-fiber rates wobble across talks, and François Chollet's rebuttal (optical information is highly compressible in ways the comparison implies it isn't) is legitimate.
The paper that actually maps onto a tree of specialists is NVIDIA's “Small Language Models are the Future of Agentic AI” — Belcak, Heinrich, Diao, Fu, Dong, Muralidharan, Lin, and Molchanov (arXiv 2506.02153, June 2025). Three pillars, verbatim: SLMs are “sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems.” Agentic workloads, they argue, mostly perform a small number of specialized tasks repetitively; paying frontier prices per call is wasteful. Where general conversation is genuinely needed, they advocate heterogeneous systems mixing SLMs and LLMs, plus an LLM-to-SLM conversion loop: log calls, cluster tasks, fine-tune small specialists to replace those calls.
Write it on the whiteboard: LeCun wants a different single architecture. NVIDIA wants a cheaper tree of today's architectures. The tree idea leans on the second.
Memory is an index, not a warehouse
One adjacent claim that holds up: memory looks a lot like an indexing graph database. The Hippocampal Memory Indexing Theory (Teyler & DiScenna, 1986; Teyler & Rudy, Hippocampus, 2007) says the hippocampus does not store content — it stores an index of the neocortical regions activated by an event. A partial cue reactivates the index; the index reactivates the distributed pattern. Pattern completion. Pointers over a distributed store. HippoRAG (Gutiérrez et al., 2024) builds long-term memory for LLMs explicitly on this theory. Modern Hopfield networks (Ramsauer et al., 2020) showed the update rule is mathematically equivalent to transformer attention. The bridge from neuroscience to RAG, GraphRAG, and GNNs is real.
The claim that “indexing slows with brain development” does not hold as stated. What the literature supports is a shift toward schema-based encoding with expertise — new information filed against structured schemas rather than raw episodes (Ghosh & Gilboa). Flag it; do not state the slowdown as fact.
The FOR case: architecture and economics
If biology is color, these numbers are the load. The strongest hard evidence for the tree is not a frog's retina. It is sparse activation, routing cascades, and orchestrator-worker agents.
MoE is the tree already inside the model
Sparse Mixture-of-Experts means only a fraction of parameters fire per token. DeepSeek-V3: 671B total parameters, 37B activated per token (~5.5%). Fine-grained routed experts plus always-on shared experts, plus auxiliary-loss-free load-balancing — a learnable per-expert bias nudged when an expert is over- or under-loaded. Trained on 14.8T tokens for ~2.788M H800 GPU-hours. Its predecessor DeepSeek-V2 was 236B total / 21B active.
Switch Transformer (Fedus, Zoph & Shazeer, 2021): up to 1.6 trillion parameters, “up to 7× increases in pre-training speed with the same computational resources” over T5-Base/Large. Mixtral 8×7B (Mistral, Dec 2023): 46.7B total, 12.9B per token — “processes input and generates output at the same speed and for the same cost as a 12.9B model.” GShard (Lepikhin et al., 2020): >600B parameters; the dense equivalent would have cost “more than ten times to train” while trailing in quality.
Note the dual-use fact: routing collapse and load imbalance (Shazeer et al., 2017) are why DeepSeek needed that balancing trick in the first place. Routing is hard even inside one model. Cite it for both sides.
Routing and cascades pay the bills
FrugalGPT (Chen, Zaharia & Zou, Stanford, 2023): an LLM cascade — cheap model first, escalate when quality thresholds fail — with up to 98% cost reduction while matching or beating GPT-4 on benchmarks like HEADLINES. Their estimate: 50–90% of naïve inference spend is wasteful.
RouteLLM (Ong et al., LMSYS / UC Berkeley, ICLR 2025): routers learned from Chatbot Arena preference data. The matrix-factorization router hits 95% of GPT-4 performance using only 14% strong-model calls — about 75% cost reduction versus random routing on MT-Bench, ~45% on MMLU, ~35% on GSM8K. The savings are benchmark-specific. That is itself an honest point about routing: difficulty skew is the whole game. Related work — Hybrid LLM, MixLLM, AutoMix — typically lands 40–85% savings depending on how skewed query difficulty is.
Small models still win narrow tasks. MedGemma-4B fine-tuned beat zero-shot GPT-4 on medical image disease classification: 80.37% vs 69.58% mean accuracy (arXiv 2512.23304). SLM edge economics are stark — Groq-served Llama has been quoted around $0.05 per million input tokens. Hold that thought; §5 has a counterpunch.
Orchestrator-worker is the flagship result
Anthropic's June 2025 engineering post on their multi-agent research system is the result everyone cites, so cite it carefully. Pattern: a lead agent (Claude Opus 4) plans and spawns 3–5 subagents (Claude Sonnet 4), each with its own context window, plus a separate citation-checking pass. It outperformed single-agent Claude Opus 4 by 90.2% on Anthropic's internal research eval. Cost: ~15× more tokens than a normal chat. On BrowseComp, token usage alone explained 80% of performance variance; tool calls (~10%) and model choice (~5%) brought the explained variance to ~95%. These figures are from Anthropic's own blog and have not been independently reproduced — label them as such.
Crucially, Anthropic scopes the pattern themselves: domains that require all agents to share the same context, or that involve many dependencies between agents, “are not a good fit for multi-agent systems today.” Intellectual ancestor: Marvin Minsky's Society of Mind (1986). Production frameworks: AutoGen, CrewAI, LangGraph supervisor, MetaGPT, ChatDev.
The mechanistic heart of “architect reviews the worker” is verifier–generator asymmetry. Best-of-N with a reward model, self-consistency, process reward models — checking a candidate is empirically cheaper than producing it. Pattern, not theorem. Still the thing that makes the tree economically coherent.
The AGAINST case — steelmanned, no softballs
If you skip this section, a senior reader will skip the rest of you.
The Bitter Lesson
Rich Sutton, 2019: “General methods that leverage computation are ultimately the most effective, and by a large margin.” Across chess, Go, speech, vision, and NLP, hand-crafted human knowledge repeatedly lost to general methods plus scale. Sutton won the 2024 Turing Award (announced 2025), which renews the argument's authority. The sharp form of the anti-thesis: your carefully architected tree of specialists is exactly the kind of hand-crafted structure scale eventually eats. Note what Sutton did not say: “Scale Is All You Need.” He said general methods that leverage computation win. MoE — learned sparse routing inside one training objective — is more Bitter-Lesson-compatible than a hand-built multi-agent org chart. Hold that distinction; it matters at the end.
Generalist eats specialist
The single hardest data point against the naïve tree is Nori et al., “Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine” (Microsoft, arXiv 2311.16452, 2023). GPT-4 with Medprompt — general-purpose prompting, no medical fine-tuning — beat Google's fine-tuned specialist Med-PaLM 2 across all nine MultiMedQA benchmarks, scoring 90.2% on MedQA versus Med-PaLM 2's 86.5% (a 27% error-rate reduction; first model over 90%). A generalist overtook a purpose-built specialist on the specialist's home turf.
Counter-counterpoint, because honesty is the brand: MedGemma-4B fine-tuned still beats zero-shot GPT-4 on some image tasks. The outcome is task- and time-dependent. Do not overgeneralize in either direction.
Multi-agent systems fail with hard numbers
Cemri, Pan, Yang et al. (UC Berkeley), “Why Do Multi-Agent LLM Systems Fail?” (arXiv 2503.13657; NeurIPS 2025). They built MAST — Multi-Agent System Failure Taxonomy — from 1,600+ annotated traces across seven popular MAS frameworks (taxonomy derived from 200+ traces; inter-annotator κ = 0.88). Result: 14 failure modes in three categories — System Design / specification issues 41.77%, Inter-Agent Misalignment 36.94%, Task Verification 21.30%. Their blunt headline: multi-agent “performance gains on popular benchmarks are often minimal,” and most failures stem from system design, not model quality. Error compounding, context loss between agents, coordination overhead — systemic, not incidental.
Cognition's Walden Yan, June 2025, “Don't Build Multi-Agents,” is the strongest practitioner argument against decomposition. Parallel subagents make implicit choices about style, edge cases, and code that later conflict; each receives only the explicit objective, not the full implicit context; “on the surface it looks like collaboration; underneath it is often parallelized guesswork.” The lead agent ends up cleaning up conflicts between partially compatible drafts. Prescription: single-threaded agents plus a dedicated compression LLM, with context engineering as “effectively the #1 job of engineers building AI agents.”
Honesty note: in March 2026 Cognition shipped “Devin can now Manage Devins” — a coordinator that scopes work and assigns pieces to isolated managed Devins. A partial architectural concession that some decomposition works. Use the nuance; neither essay is gospel.
Routing is itself the bottleneck
Mis-routing, the near-impossibility of estimating task difficulty a priori, and the fact that the router must be as capable as the hardest routing decision — these are not edge cases. GPT-5 launched August 7, 2025 with a “real-time router” between a smart efficient model and a deeper reasoning model. OpenAI's own description: the router decides based on conversation type, complexity, tool needs, and explicit intent, continuously trained on real signals including model switches and preference rates. On day one the autoswitcher broke. Sam Altman: “Yesterday, the autoswitcher broke and was out of commission for a chunk of the day, and the result was GPT-5 seemed way dumber.” Critically for this thesis, OpenAI's stated end-goal cuts against permanent decomposition: “In the near future, we plan to integrate these capabilities into a single model.” The tree collapsing back into the trunk.
Maintenance burden and the end-to-end counter-trend
N specialists = N training pipelines, N eval suites, N drift problems, N versioning headaches. The SLM and agent papers systematically underweight this. Practitioners do not.
And then there is the trend the tree camp most needs to answer: end-to-end learning has repeatedly killed hand-engineered pipelines. Speech recognition: phoneme/HMM stacks → end-to-end deep nets. Tesla FSD v12: city-streets stack upgraded “to a single end-to-end neural network… replacing over 300k lines of explicit C++ code” (release notes); Musk on the Q4 2023 earnings call (Jan 24, 2024): “We replaced 330,000 lines of C++ code with neural nets.” Tesla AI lead Ashok Elluswamy on why they abandoned modular: “We used to work on an explicit, modular approach because it was so much easier to debug. But what we found out was that codifying human values was really difficult.”
Wayve's founding thesis (“AV2.0”) was that one large neural network would outperform modular robotics stacks; they adapted a UK-trained model to US right-side driving with ~500 hours of incremental data over eight weeks. The sophisticated counterweight is Waymo. They published EMMA (end-to-end on Gemini, Oct 2024) but do not deploy pure end-to-end. Co-CEO Dmitri Dolgov: a monolithic architecture “makes it very easy to get started, but it's wildly inadequate to go to full autonomy safely and at scale.” Waymo's December 2025 Foundation Model post describes a hybrid that keeps module boundaries for debuggability and independent validation while backpropagating end-to-end. That is the most mature real-world answer I have found. This post should adopt its spirit.
One more threat to name: long context. As windows grow from 100K to 1M+ tokens and in-context memory improves, part of the retrieval/specialist-decomposition rationale weakens — you can increasingly just put everything in one model's context. If you are building a RAG empire on the assumption that context will always be scarce, write that assumption down and date it.
What the industry is actually shipping
Strip the philosophy and look at production. The dominant pattern in 2025–2026 is the compound AI system. Zaharia, Khattab, Chen, Davis, Miller, Potts, Zou, Carbin, Frankle, Rao, and Ghodsi (Berkeley BAIR, Feb 18, 2024): “a system that tackles AI tasks using multiple interacting components, including multiple calls to models, retrievers, or external tools.” Verbatim datapoint: Databricks colleagues found 60% of LLM applications use some form of RAG, and 30% use multi-step chains. Their thesis — now mainstream — is that leading results can be achieved through clever engineering, not just scaling training, and that compound systems “will remain a leading paradigm even as models improve.” That is the strongest institutional endorsement of the tree.
Production menus make the economic case in one number. Across current model families there is roughly a ~100× price spread between the cheapest usable tier and the flagship reasoning tier. Claude tiers (Haiku / Sonnet / Opus), Gemini tiers (Flash-Lite / Flash / Pro), OpenAI's mini-to-reasoning ladder — exact per-token prices move monthly and third-party aggregators lag the vendor pages, so verify before you put a dollar figure in a deck. The structural pattern is robust: a steep price gradient is why routing exists. It is the entire economic case for the tree in one ratio.
The counter-current is well-funded and serious. LeCun's world- models bet at AMI Labs and Fei-Fei Li's World Labs represent the view that the future is a different single architecture, not a tree of today's models. Do not confuse “compound systems are winning in production today” with “compound systems are the final form of intelligence.”
The conditional conclusion
Here is the position I can defend in a room with people who have read the papers:
A tree of models wins when (i) tasks decompose cleanly into independent subtasks, (ii) verification is cheaper than generation, (iii) query difficulty is skewed enough that routing captures cheap wins, and (iv) subtasks do not need shared implicit context. Monoliths and end-to-end win when the problem is entangled, when context-sharing is essential, or when the whole task already fits one model's context window.
That mirrors Waymo's hybrid and Anthropic's own scoping. It is a stronger, more credible position than “the tree wins, full stop.”
Be explicit about which tree you mean. MoE is a learned sparse routing structure inside one model with one training objective — the Bitter-Lesson-compatible version of the thesis. A hand-built multi-agent system is engineered decomposition — the version where MAST's 14 failure modes concentrate and where Sutton's warning has teeth. The author's strongest ground is that the learned tree is winning. The hand-built tree is where the failure evidence lives.
Concrete thresholds that would change the recommendation:
- If frontier context windows and in-context memory keep improving so that whole workflows fit one context at falling cost, decomposition's ROI shrinks — revisit RAG and specialist splits.
- If a single unified router-inside-the-model (OpenAI's stated GPT-5 end-goal) subsumes external multi-agent orchestration, the “tree” migrates inside the model and your external orchestration layer thins out.
- If, on your specific tasks, multi-agent failure rates (MAST-style evals) do not beat a single strong agent, do not decompose. The coordination tax is not worth it.
The cover image is the argument in one frame: a billion sensory threads braid into a cord, pass through a needle, and leave one line on the sand — while a surreal tree of eyes and ears stands over the desert like an architect that never touches the raw feed. The picture is right. The temptation is to treat the picture as a proof. It isn't. The proof, such as it is, lives in sparse activation ratios, cascade cost curves, and the uncomfortable fact that verification keeps being cheaper than generation — until the day the problem stops decomposing and you need the titan after all.
Caveats
- The neuroscience (retinal bandwidth ~10⁷ bits/s; ~10 bits/s conscious throughput; ≥30 RGC channels; ~10 orders of magnitude light adaptation; hippocampal indexing; predictive coding) is peer-reviewed and well-established. The inference that this vindicates AI decomposition is the author's analogy.
- LeCun's bandwidth numbers are internally inconsistent across talks; present the ~50× figure as rhetorical estimate.
- Specific AMI Labs seed/valuation figures circulating in secondary tech outlets are reported, not confirmed — I have omitted dollar amounts for that reason.
- Anthropic's 90.2% quality gain and 15× token figures are from an internal eval on an engineering blog.
- Claude/Gemini pricing figures move with model versions; verify against vendor pages. The ~100× cheapest-usable-to-flagship structural pattern is the real point.
- Generalist-vs-specialist results are task- and time-dependent: Medprompt (2023) showed a generalist winning; MedGemma (2025) shows a fine-tuned small specialist still beating zero-shot GPT-4 on some tasks.
References
Neuroscience
- Zheng & Meister (2024) — The unbearable slowness of being: Why do we live at 10 bits/s? Neuron. DOI: 10.1016/j.neuron.2024.11.008
- Koch et al. (2006) — How Much the Eye Tells the Brain. Current Biology 16. DOI
- Lettvin, Maturana, McCulloch & Pitts (1959) — What the Frog's Eye Tells the Frog's Brain. IEEE
- Gollisch & Meister (2010) — Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina. Neuron 65. DOI
- Baden et al. (2016) — The functional diversity of retinal ganglion cells in the mouse. Nature. DOI
- Fain et al. (2001) — Adaptation in Vertebrate Photoreceptors. Physiological Reviews. DOI
- Rao & Ballard (1999) — Predictive coding in the visual cortex. Nature Neuroscience 2. DOI
- Teyler & Rudy (2007) — The hippocampal indexing theory and episodic memory. Hippocampus. DOI
Architecture, routing, agents
- Belcak et al. (2025) — Small Language Models are the Future of Agentic AI. arXiv:2506.02153
- DeepSeek-AI — DeepSeek-V3 technical report (671B / 37B active). arXiv:2412.19437
- Fedus, Zoph & Shazeer (2021) — Switch Transformers. arXiv:2101.03961
- Chen, Zaharia & Zou (2023) — FrugalGPT. arXiv:2305.05176
- Ong et al. (2025) — RouteLLM. ICLR 2025. arXiv:2406.18665
- Anthropic (June 2025) — Engineering post on multi-agent research system (+90.2% / ~15× tokens). Anthropic Engineering
- Zaharia et al. (2024) — The Shift from Models to Compound AI Systems. Berkeley BAIR. BAIR blog
- Gutiérrez et al. (2024) — HippoRAG. arXiv:2405.14831
- Ramsauer et al. (2020) — Hopfield Networks is All You Need. arXiv:2008.02217
The steelman
- Sutton (2019) — The Bitter Lesson. incompleteideas.net
- Nori et al. (2023) — Can Generalist Foundation Models Outcompete Special-Purpose Tuning? arXiv:2311.16452
- Cemri et al. (2025) — Why Do Multi-Agent LLM Systems Fail? arXiv:2503.13657
- Yan / Cognition (June 2025) — Don't Build Multi-Agents. Cognition blog
- LeCun (2022) — A Path Towards Autonomous Machine Intelligence. OpenReview
On this site
- One Model to Rule Them All — the tabular sibling of this argument: trees before titans.
- Anatomy of an Agentic AI System — routing, retrieval, and when specialists actually earn their keep.