Most DPO datasets are pairs: prompt + good response vs bad response. That’s binary thinking. Laeka proposes the Correction Triangle: prompt + flawed response WITH DIAGNOSIS + superior response WITH EXPLANATION.<\/p>\n
The diagnosis matters. When an LLM learns why a response fails\u2014missing nuance, logical gap, ethical lapse\u2014it doesn’t just memorize “prefer this one.” It internalizes the structure of better thinking.<\/p>\n
Standard DPO treats preference as a black box. The model learns patterns but not principles. Add diagnosis and you’re teaching reasoning about reasoning. The inferior response comes with a reason it’s inferior. The superior response explains the correction.<\/p>\n
This produces stronger training signal. Models trained on diagnostic DPO show better generalization to novel prompts. They don’t overfit to surface-level patterns.<\/p>\n
Each triangle consists of three elements:<\/p>\n
1. Prompt:<\/strong> The original instruction or question.<\/p>\n 2. Flawed Response + Diagnosis:<\/strong> A response that fails, plus structured annotation of why\u2014missing key information, logical inconsistency, tone mismatch, scope creep.<\/p>\n 3. Superior Response + Explanation:<\/strong> The better answer, annotated with the principle or reasoning that makes it superior.<\/p>\n Prompt:<\/strong> “Explain quantum entanglement to a high school student.”<\/p>\n Flawed + Diagnosis:<\/strong> “Two particles become linked so they affect each other instantly across any distance.” [Diagnosis: Oversimplifies; creates false impression of faster-than-light communication; misses the philosophical weirdness that makes entanglement interesting.]<\/p>\n Superior + Explanation:<\/strong> “Quantum entanglement means two particles can be correlated in a way that classical physics can’t explain. Measuring one instantly affects what you know about the other\u2014but you can’t use this to send information faster than light. The weirdness is that this correlation seems to exist even though nothing physical travels between them.” [Explanation: Addresses the core mystery; clarifies the common misconception about superluminal signaling; invites wonder rather than just stating facts.]<\/p>\n The Correction Triangle turns preference data into reasoning data. Every pair is now a teaching moment. The model learns not just what’s good but how good emerges from understanding.<\/p>\n This aligns with cognitively integrated AI principles: training through clarity, diagnosis, and explanation rather than brute-force preference optimization.<\/p>\nConcrete Example<\/h2>\n
Why Laeka Chose This<\/h2>\n