REINFORCE with a baseline, then PPO clipping

Train a policy-gradient agent on a small control task over several seeds, add a learned baseline, then the PPO clipped objective, and compare stability.

The exercise happens on Self-directed project (Spinning Up algorithm docs as reference). Canonical link: https://spinningup.openai.com/en/latest/algorithms/vpg.html. Link check: link responded (5 Oct 2026). Opened from this app 0 times; opening is not counted as an attempt.

Supports: Hands-on understanding of the RL machinery behind RLHF.