Controlled Hallucination: Anil Seth Was Talking About LLMs Too

Anil Seth didn’t set out to describe how language models work. He was describing the human brain. But the parallels are so precise they border on uncomfortable.

Perception as Prediction

Seth’s core argument is elegant. Your brain doesn’t receive reality — it generates reality. Every moment of conscious experience is a prediction, refined by sensory input. You don’t see the world as it is. You see your brain’s best guess about the world, updated just enough to keep you alive.

This is controlled hallucination. The emphasis is on “controlled.” Without the control — without sensory input constraining the predictions — you get actual hallucinations. Dreams. Psychosis. States where the generative engine runs without adequate feedback.

Now read that paragraph again, replacing “brain” with “language model” and “sensory input” with “training data and prompt context.” The structural mapping is exact.

The Generative Engine Is the Same

A transformer generates token predictions. Each token is the model’s best guess about what comes next, given everything before it. The training data acts like developmental experience — shaping the priors. The prompt acts like current sensory input — constraining the prediction.

When the constraints are strong (specific prompt, well-represented topic, clear context), the model’s outputs closely match reality. When the constraints are weak (vague prompt, rare topic, ambiguous context), the generative engine fills in the gaps with plausible-sounding content that may not be true.

This isn’t a flaw in the architecture. This is the architecture. The same mechanism that produces accurate, helpful responses also produces hallucinations. The variable isn’t the engine — it’s the quality of the constraints.

Predictive Processing and the Free Energy Principle

Seth’s work builds on Karl Friston’s free energy principle: biological systems minimize surprise by maintaining accurate predictive models of their environment. The brain constantly updates its generative model to reduce the gap between prediction and reality.

Language model training does exactly this. The loss function measures the gap between the model’s predictions and the actual next token. Training minimizes this gap across billions of examples. The result is a generative model that, like the brain, produces predictions that usually match reality — but sometimes don’t.

The critical insight is that prediction error is the signal. In neuroscience, prediction errors drive learning and attention. In language models, they drive gradient updates. The math is different but the principle is identical: surprise is information, and systems learn by being wrong.

Where the Analogy Gets Practical

If language models are controlled hallucination engines, then the alignment question becomes: how do you improve the control without killing the generation?

In Seth’s framework, control comes from sensory grounding. The brain stays anchored to reality through continuous feedback from the body and the world. Disruptions to this grounding — sensory deprivation, psychedelic drugs, neurological conditions — produce uncontrolled hallucination.

Language models lack this continuous grounding. They generate in open loop — producing output without real-time feedback about whether that output corresponds to reality. Retrieval-Augmented Generation (RAG) is essentially an attempt to add sensory grounding: anchor the model’s predictions to verified external data.

But RAG is crude compared to the brain’s feedback mechanisms. The brain integrates millions of sensory signals simultaneously, at multiple levels of abstraction, with microsecond latency. Current grounding techniques are more like occasionally checking a fact sheet.

Contemplative Practice as Enhanced Control

Here’s where contemplative practice enters. Meditation doesn’t stop the brain’s generative engine. It enhances the monitoring system. An experienced meditator has better real-time awareness of their own predictions, better detection of when those predictions are ungrounded, and better ability to flag uncertainty.

This is the missing piece in current AI alignment. We need models that don’t just generate — they need to monitor their own generation. Not through external fact-checking, but through internal mechanisms that track the quality of the prediction in real-time.

Some research moves in this direction. Confidence calibration work tries to align a model’s stated certainty with its actual accuracy. But most of this work is post-hoc — analyzing outputs after generation, not monitoring the generation process itself.

The Research Agenda

Seth’s framework suggests three concrete research directions for AI alignment. First, better grounding mechanisms — not just RAG, but continuous, multi-level feedback during generation. Second, internal monitoring — training models to detect when their own predictions are weakly supported. Third, uncertainty communication — models that naturally express their confidence level as part of the generation process, not as an afterthought.

The contemplative tradition has been developing internal monitoring techniques for millennia. The neuroscience of meditation is finally explaining why these techniques work. And the structural parallels to language models are too clear to ignore.

At Laeka Research, we’re translating these insights into practical training methodologies. Anil Seth described the architecture of conscious experience. It turns out he also described the architecture of language generation. The question now is what we do with that knowledge.