Reward Modeling

Describe how preference data becomes a learned reward, how that reward is evaluated, and how optimization can exploit its errors.