Distillation

Also called: model distillation, knowledge distillation, distilled model, teacher student

Training a smaller model to copy the outputs of a larger one, so it keeps much of the quality while being cheaper and faster to run.

For example: A lab uses a large model's answers to train a small model that runs on phones.