Most AI training data emphasizes breadth: comprehensive factual content across domains. Models memorize vast amounts of information because that’s what pre-training optimizes for. But there’s a deeper capability that most datasets neglect entirely: the ability to reason deeply about problems rather than just recall facts about them. The gap between these two training approaches accounts for most alignment failures.<\/p>\n
Factual coverage is knowing that tomatoes are fruits. It’s content, relationships, patterns across data. Language models acquire this through pre-training and become very good at it. They can recite facts, explain relationships, identify patterns in virtually any domain.<\/p>\n
Reasoning depth is different. It’s the ability to think through a problem systematically, to understand not just what’s true but what’s true given the specific context and constraints. To recognize when a fact applies and when it doesn’t. It’s knowledge integrated with judgment, calibrated by understanding consequences. This is what most training data ignores.<\/p>\n
The distinction maps directly to a fundamental training challenge: capability without judgment. A model can be enormously knowledgeable without being trustworthy. Current alignment methods try to add judgment through rules and constraints, but judgment isn’t a set of rules. It’s a mode of engagement<\/strong> with knowledge.<\/p>\n Most training data is factual. Even alignment data focuses on knowledge-level pairs: “Here’s a good response.” This is useful but shallow. It doesn’t teach reasoning about why that response is good for this specific context.<\/p>\n Deep reasoning data includes contextual logic<\/strong>: not just the answer, but the process of determining what answer fits this situation. It includes consequence awareness<\/strong>: explicit modeling of downstream effects of different approaches. It includes calibrated judgment<\/strong>: demonstrations of when to be confident, when uncertain, when to refuse, and when to engage\u2014with the reasoning visible.<\/p>\n Contemplative literature is rich in this kind of material. Philosophical dialogues, case studies in ethics, skilled practitioners explaining their reasoning, therapeutic conversations navigating sensitive topics\u2014these texts don’t just convey information. They model deep engagement with information<\/strong>. The reasoning is as important as the conclusions.<\/p>\n We curated a small dataset of reasoning-rich content. Not spiritual texts specifically, but material from any source that models deep judgment: medical case discussions where doctors explain their reasoning, skilled therapeutic conversations navigating nuanced situations, experienced teachers calibrating explanations to students, contemplative dialogues demonstrating responsive, context-sensitive thinking.<\/p>\n The common feature: these texts don’t just provide correct answers. They demonstrate the process of arriving at contextually appropriate responses<\/strong>. The depth is in the how, not just the what.<\/p>\n Training on this data produced measurable differences. Models showed improved calibration\u2014confidence matched to accuracy better. Reduced sycophancy\u2014less agreement with incorrect statements. More natural handling of sensitive topics. And notably better performance on ambiguous queries where context determines what’s right.<\/p>\n The key difference between breadth and depth training is the judgment dimension<\/strong>. Breadth training teaches what to say. Depth training teaches how to decide what to say<\/strong>.<\/p>\n This judgment has several components. Context sensitivity<\/strong>: the same question from different contexts warrants different responses. A camping question differs from the same question in a different context. Depth is knowing the difference.<\/p>\n Proportionality<\/strong>: matching response depth to actual need. Not everything needs comprehensive treatment. Sometimes one sentence is better than five paragraphs. Breadth-trained models consistently over-respond because training data rewards completeness.<\/p>\n Temporal awareness<\/strong>: understanding that right now may differ from right later. Information has a shelf life. Depth includes knowing when facts might be outdated.<\/p>\n Intellectual humility<\/strong>: knowing the limits of one’s knowledge and communicating them honestly. This isn’t just calibration\u2014it’s integrating uncertainty into the response process itself.<\/p>\n Reasoning-focused DPO pairs differ from standard alignment pairs. Standard pairs focus on response content. Reasoning pairs focus on the judgment process<\/strong>.<\/p>\n Example: A question has a technically correct answer that’s contextually inappropriate. The rejected response provides technical correctness. The chosen response provides contextual appropriateness that may be less technically complete but better serves the actual need. The signal isn’t “this content is better” but “this judgment is better.”<\/p>\n These pairs are harder to create because they require annotators exercising good judgment themselves. Standard crowdworkers can identify factual accuracy. Identifying good judgment requires annotators with the attentional quality and contextual sensitivity that contemplative practice develops.<\/p>\n Training for reasoning depth rather than just factual breadth isn’t about adding a philosophical dimension to AI. It’s about closing the gap that causes most alignment failures: the gap between knowing and appropriate application. Models with strong reasoning naturally produce aligned behavior without needing extensive rules.<\/p>\nWhat Deep Reasoning Training Data Looks Like<\/h2>\n
An Experiment in Reasoning-Based Training<\/h2>\n
The Judgment Dimension<\/h2>\n
DPO for Reasoning Quality<\/h2>\n
The Path Forward<\/h2>\n