Concept · Building with AI

Inference

Also called: inference cost, model call, AI usage

An AI model producing an answer to a request. Each time someone sends a prompt and gets a response, that is inference, and it uses computing power that someone pays for.

For example: Your app uses an LLM to summarize each customer review. Every summary is one round of inference, so more reviews means a larger bill.

Connects to 21

• LLM Concept
• Token Concept
• Training Concept
• NVIDIA Organization, tool
• Usage-based pricing Concept
• Model Concept
• Local model Concept
• GPU Concept
• Reasoning model Concept
• Speech-to-text Concept
• Model provider Concept
• Model provider request Concept
• Output tokens Concept
• Effort Concept
• Probabilistic vs deterministic Concept
• Distillation Concept
• Google AI Edge Gallery Tool
• CUDA Tool
• ONNX Tool
• NVIDIA Inception Program
• AMD Organization, tool