# Inference

Concept · [Building with AI](https://ecosystem.samcarlton.com/ai)

Also called: inference cost, model call, AI usage

An [AI model](https://ecosystem.samcarlton.com/model) producing an answer to a request. Each time someone sends a [prompt](https://ecosystem.samcarlton.com/prompt) and gets a response, that is inference, and it uses computing power that someone pays for.

**For example:** Your app uses an [LLM](https://ecosystem.samcarlton.com/llm) to summarize each customer review. Every [summary](https://ecosystem.samcarlton.com/secondary-source) is one round of inference, so more reviews means a larger bill.

## Connects to (21)

- [LLM](https://ecosystem.samcarlton.com/llm): Large language model.
- [Token](https://ecosystem.samcarlton.com/token): In AI, a small chunk of text, roughly a word or a piece of a word, that an LLM reads and writes.
- [Training](https://ecosystem.samcarlton.com/training): Teaching an AI model by showing it a very large number of examples before anyone uses it.
- [NVIDIA](https://ecosystem.samcarlton.com/nvidia): NVIDIA makes GPUs, chips that help computers train AI models and run them to produce answers.
- [Usage-based pricing](https://ecosystem.samcarlton.com/usage-based-pricing): A way to charge where the bill follows how much the customer uses, instead of a flat fee.
- [Model](https://ecosystem.samcarlton.com/model): The trained system at the center of an AI product.
- [Local model](https://ecosystem.samcarlton.com/local-model): A model that is downloaded and run on a person's own computer or a company's own servers, not on a provider's.
- [GPU](https://ecosystem.samcarlton.com/gpu): Graphics processing unit.
- [Reasoning model](https://ecosystem.samcarlton.com/reasoning-model): A model, or a mode of a model, that works through a problem in intermediate steps before it answers.
- [Speech-to-text](https://ecosystem.samcarlton.com/speech-to-text): Software that turns spoken words into written text.
- [Model provider](https://ecosystem.samcarlton.com/model-provider): The company or service that runs an AI model and answers requests to it, usually over the internet through an API.
- [Model provider request](https://ecosystem.samcarlton.com/model-provider-request): One trip from an AI product to the model provider and back.
- [Output tokens](https://ecosystem.samcarlton.com/output-tokens): The tokens the model writes back.
- [Effort](https://ecosystem.samcarlton.com/effort): A setting that tells a reasoning model how much thinking to do before it answers.
- [Probabilistic vs deterministic](https://ecosystem.samcarlton.com/probabilistic-vs-deterministic): Deterministic software gives the same output for the same input every time.
- [Distillation](https://ecosystem.samcarlton.com/distillation): Training a smaller model to copy the outputs of a larger one, so it keeps much of the quality while being cheaper and faster to run.
- [Google AI Edge Gallery](https://ecosystem.samcarlton.com/google-ai-edge-gallery): A free Google app for phones that downloads open AI models and runs them on the device, with no internet connection needed after download.
- [CUDA](https://ecosystem.samcarlton.com/cuda): NVIDIA's platform for writing software that runs on its GPUs.
- [ONNX](https://ecosystem.samcarlton.com/onnx): An open file format for AI models, so a model trained with one tool can run with another, on servers, phones or in a browser.
- [NVIDIA Inception](https://ecosystem.samcarlton.com/nvidia-inception): NVIDIA's free program for AI startups.
- [AMD](https://ecosystem.samcarlton.com/amd): A chip company that makes processors for computers and servers and AI accelerators for data centers.

Map view: https://ecosystem.samcarlton.com/inference