# Distillation

Concept · [Building with AI](https://ecosystem.samcarlton.com/ai)

Also called: model distillation, knowledge distillation, distilled model, teacher student

Training a smaller model to copy the outputs of a larger one, so it keeps much of the quality while being cheaper and faster to run.

**For example:** A lab uses a large model's answers to train a small model that runs on phones.

## Connects to (5)

- [Training](https://ecosystem.samcarlton.com/training): Teaching an AI model by showing it a very large number of examples before anyone uses it.
- [Quantization](https://ecosystem.samcarlton.com/quantization): Shrinking a model by storing its weights with fewer digits of precision.
- [Model](https://ecosystem.samcarlton.com/model): The trained system at the center of an AI product.
- [Inference](https://ecosystem.samcarlton.com/inference): An AI model producing an answer to a request.
- [Fine-tuning](https://ecosystem.samcarlton.com/fine-tuning): Extra training of an existing model on a smaller, narrower set of examples, so it suits one task or style.

Map view: https://ecosystem.samcarlton.com/distillation