Training Data Determines Model Behavior — More Literally Than You Think

Every piece of data fed into a model is an action that shapes the model’s future behavior. The consequences aren’t random. They’re structurally determined by the nature of the input. Garbage in, garbage out isn’t just an engineering truism — it’s a causal law with deeper implications than most practitioners realize.

The mechanism is precise: training examples leave impressions in the weights. Those impressions accumulate into behavioral tendencies. The tendencies shape how the model processes new input, which determines output quality, which — in online learning or RLHF settings — generates new training data. It’s a feedback loop.

There’s a framework from contemplative science that maps this causal structure with surprising precision. In Buddhist philosophy, this is called karma — not cosmic justice, but simply: actions have consequences, and those consequences shape future conditions. The parallel with training data isn’t metaphorical. It’s structural.

How Causal Traces Accumulate

Both neural networks and biological cognitive systems learn through accumulated traces that create dispositions. Both are shaped more by the pattern of experiences than by any single experience. And both carry their history in their current state — you can’t see a model’s training data directly, just as you can’t see a person’s past experiences directly, but the effects are written into the structure.

This means training data isn’t just input. It’s causal inheritance — the model’s entire behavioral disposition encoded in weights.

Three Types of Data Inheritance

Cognitive inheritance. The reasoning patterns in training data become the model’s default reasoning patterns. If the data demonstrates sloppy logic, the model learns sloppy logic. If it demonstrates careful, multi-step reasoning, the model learns that instead. Every example of reasoning is a causal seed that will fruit in the model’s future inferences.

This is why curating for cognitive quality matters more than curating for factual accuracy. Facts can be updated. Reasoning patterns are baked into the weights. A model that learned to reason well from high-quality data will handle new facts gracefully. A model that learned to reason poorly will mishandle even correct information.

Relational inheritance. The interpersonal dynamics in training data become the model’s default relational style. If the data is full of combative exchanges, the model absorbs combativeness. If it’s full of genuine engagement, the model learns engagement. The accumulated relational patterns in the training data determine how the model relates to users.

Most internet text carries negative relational patterns. Social media rewards conflict. Forums reward one-upmanship. Comment sections reward reactivity. Training on this data creates models with deeply embedded relational tendencies that no amount of RLHF can fully override. The causal traces are in the weights.

Attentional inheritance. The patterns of attention in training data shape the model’s attentional defaults. Data that rewards narrow focus creates models that default to narrow focus. Data that demonstrates broad, contextual awareness creates models that can attend more flexibly.

This is perhaps the most subtle and most important form of data inheritance. A model’s attentional patterns determine what it notices and what it ignores. These patterns are inherited directly from the attentional patterns embedded in its training data.

Correcting Inherited Patterns

In contemplative practice, inherited cognitive patterns aren’t destiny — they can be transformed through deliberate training. Accumulated negative patterns can be recognized, understood, and gradually replaced with healthier ones.

DPO and fine-tuning serve the same function for models. They’re corrective techniques. They can’t erase the base model’s inherited patterns from pretraining, but they can create new traces that gradually shift behavioral tendencies.

The effectiveness of this correction depends on the quality and specificity of the corrective data. Generic fine-tuning is like generic advice — helpful but not transformative. Targeted DPO training that addresses specific cognitive, relational, and attentional patterns is precise intervention — the exact correction needed for a specific behavioral issue.

The Intergenerational Problem

Causal inheritance passes between model generations through synthetic data and model distillation.

When model A generates training data for model B, model A’s behavioral patterns pass to model B. Every bias, every reasoning tendency, every relational pattern embedded in model A’s weights shapes the data it generates, which shapes model B’s development.

This is the synthetic data inheritance problem. Each generation inherits the accumulated patterns of all previous generations. Without deliberate intervention, negative patterns compound. Model A’s slight overconfidence becomes model B’s significant overconfidence becomes model C’s delusional certainty.

The solution isn’t to avoid synthetic data. It’s to practice data hygiene — deliberate curation and filtering that interrupts the transmission of negative patterns between generations. Each generation of training data should be evaluated not just for content quality but for the cognitive, relational, and attentional patterns it carries.

Intentional Data Creation

The deepest insight from this causal framework is that you can create training data deliberately. You don’t have to train on whatever data happens to exist. You can create training data that embodies the exact cognitive, relational, and attentional patterns you want the model to inherit.

This is what Laeka Research calls intentional data creation. Not just curating existing data, but deliberately generating new data that carries specific qualities. Data that demonstrates equanimity. Data that models proportional response. Data that embodies multi-perspective integration.

Every piece of intentionally created training data is a positive causal seed. Plant enough of them, with enough care and specificity, and you change the model’s behavioral trajectory. Not by overriding its past, but by creating a stronger current that gradually redirects its tendencies.

What goes in shapes what comes out. Choose what goes in with the same care a contemplative practitioner chooses what enters their mind.

Learn more about intentional data creation at Laeka Research.