The quality of AI training data is the biggest bottleneck in alignment research. Most DPO and RLHF datasets are generated by crowdworkers operating under time pressure, with vague guidelines and minimal cognitive training. The…
Current AI systems think in binaries. True or false. Positive or negative. Safe or unsafe. This works for classification tasks. For anything that matters, it fails. The limitation lies in how models are forced…
Anil Seth didn’t set out to describe how language models work. He was describing the human brain. But the parallels are so precise they border on uncomfortable. Perception as Prediction Seth’s core argument is…
Attention isn’t about emptying the mind. It’s about watching the mind do its thing — and choosing not to follow every impulse. That distinction matters enormously when you’re trying to align a language model….
Every few months, someone publishes a paper claiming neural networks aren’t really neural. They’re mathematical functions. They’re statistical models. They’re fancy curve fitters. And technically, they’re right. But they’re missing the point entirely. A…
Transformer architectures use selective attention: focus computation on relevant tokens, filter out noise. It works, but it’s limited. Selective attention is reactive. It responds to what’s in the input without active selection based on…
Error correction in neural systems requires two things: detecting when output diverges from intent, and adjusting for context. Machine learning models struggle with edge cases because they process literal signals. A human with genuine…
Language models suffer from a fundamental pathology: they over-commit to patterns learned during training, then apply those patterns regardless of context. This is the technical core of overfitting, sycophancy, mode collapse, and a dozen…
Binary thinking forces complex situations into simple choices, discarding information. That discarded information has a cost. In computational terms, binary thinking is overhead. This applies to AI systems. It applies to human organizations. It…
The quality of an AI model depends on the quality of its training data. This is the closest thing to a universal law in machine learning. And those trained in attentional expertise produce better…