News
EveryDatacenter

Inference

Running a trained AI model to answer requests, as opposed to training it.

Also: AI inference, serving

Inference is the work of using an AI model: answering a chatbot query, writing code, recognising an image. Each request needs much less computing than training, but there are billions of them, so inference is expected to take a growing share of AI data centre capacity. Inference often benefits from being closer to users, for lower latency, and can run on a wider range of chips. Training can be done in remote places with cheap power.

See also

Find data centres that mention Inference

The weekly digest

Every Monday: the data centre of the day, new pages on EveryDatacenter, and the conferences and anniversaries of the week ahead.