RLHF

Also called: reinforcement learning from human feedback, human feedback, preference training, RLAIF

Reinforcement learning from human feedback. A training step where people compare model answers, and the model is adjusted to give the kinds of answers people preferred. It is a main way chat models learn to be helpful.

For example: Reviewers pick the clearer of two answers thousands of times, and RLHF teaches the model to write more like the preferred ones.