Loss function

Also called: loss, cost function, objective function, error function, training loss

A formula that scores how wrong a model's output is compared with the right answer. Training tries to make this number as small as possible.

For example: For next-token prediction, the loss is high when the model gave the real next word a low likelihood.