DPO loss on a toy preference problem with a known reward

Implement the DPO loss with a frozen reference model on synthetic preferences from a known reward, and trace reward against KL as beta varies.

The exercise happens on Self-directed project (authors’ reference code linked). Canonical link: https://github.com/eric-mitchell/direct-preference-optimization. Link check: link responded (5 Oct 2026). Opened from this app 0 times; opening is not counted as an attempt.

Supports: Derivation-to-code practice for post-training and alignment research interviews.