Tools

GPU vs API: what will your LLM workload cost?

Put in your traffic and prices, and compare a managed model API with running your own GPUs. Every number is editable. The presets are examples to start from, not quotes.

Workload

Requests per day

Avg input tokens

Avg output tokens

Model API

Price preset (editable example)

e.g. OpenAI gpt-6.1-sol, Anthropic Claude Sonnet 5.5. List price, standard tier.

Input price

Output price

Self-hosted GPUs

GPU preset (example on-demand rate, check your provider)

GPU hourly cost

Throughput per GPU

Throughput counts input and output tokens together, batched. The default is an example assumption: benchmark your model and serving stack.

Sustained utilisation

How busy the GPUs stay on average. Headroom for peaks lowers it.

Hosting

Always onScale to zero

GPUs (minimum fleet)

Billed 730 hours a month each. Raised automatically if the load needs more.

Caveats

Engineering and operations time is not included: deploying, patching, monitoring and on-call for your own GPUs is real cost.
Average load only. Peaks, redundancy, cold starts and minimum billing increments all push self-hosted cost up.
Throughput is the biggest unknown. It depends on the model, context length, batching and serving stack, so benchmark your own.
API prices exclude discounts, batch tiers and prompt caching, which can cut API cost a lot. Storage, networking and egress are excluded on both sides.
An open-weight model you host may not match a frontier model on quality for your task. Compare on your own data.

Not sure which route fits? Try the architecture picker or compare the consulting packages.

Presets checked 28 October 2026 against public pricing pages: OpenAI API pricing, Anthropic pricing, Lambda GPU cloud pricing. API presets are standard-tier list prices; GPU presets are single-GPU on-demand list prices from one provider. Prices change, so check your provider. A month is 730 hours.