What an AI Feature Actually Costs to Run
The build is the cheap part. Here is how to estimate the monthly bill before you commit, and the four things that quietly multiply it.
· 3 min read · AI & ML
In short: Most AI projects are budgeted like software — a build cost, then maintenance. They do not behave like that. An AI feature has a per-use cost that scales with how successful it is, and teams routinely discover this in month three.
The bill has four parts
- Tokens in. Everything you send: the instruction, the user's question, and any documents you attach for context. The last one is usually the largest and the most overlooked.
- Tokens out. What the model writes back. Cheaper to control — you can cap it — but a feature that produces long answers costs more per use than one that produces short ones.
- Retries and failures. Timeouts, rate limits, answers that fail validation and get asked again. Budget 10–20% on top in the first months.
- Everything around it. Storing documents, generating embeddings, a vector database, logging. Individually small, collectively not.
A worked example
Say a support assistant that answers from your documentation. Each question sends roughly 3,000 tokens of context plus the question, and gets back 300. That is around 3,500 tokens a conversation.
At a thousand conversations a month, that is 3.5 million tokens. On a mid-tier model, that is a modest monthly bill — tens of dollars. On a frontier model, it can be ten times that. And at fifty thousand conversations, the same architecture that was fine becomes a line item somebody asks about.
The lesson is not "use the cheap model". It is that the cost per use is fixed at design time, and multiplied later by success.
The four things that multiply it
- Stuffing the whole document in. Sending fifty pages when the answer is in one paragraph is the single most common cause of a surprising bill. Retrieve the relevant part instead.
- Using the largest model everywhere. Classifying a message as "billing" or "technical" does not need the same model as writing a legal summary. Route by difficulty.
- Re-answering identical questions. A large share of real queries repeat. Caching common answers costs nothing and removes them from the bill entirely.
- No cap on output. Without a limit, an unusual input can produce an enormous answer. Set one.
When not to use a model at all
A surprising amount of what gets specified as "AI" is a lookup, a filter or a rule. If the answer is always one of six things and the input is structured, a model is the expensive way to get a worse result. Use one where the input is genuinely messy and human language — that is what it is good at.
Before you commit
- Estimate tokens per use by hand, with a real example. It takes ten minutes and prevents most surprises.
- Multiply by your optimistic usage number, not your conservative one. Plan for it working.
- Decide the ceiling you would be uncomfortable crossing, and put a hard limit in before launch, not after.
- Log every call with its token count from day one. Without this you cannot tell a cost problem from a usage problem.
We build AI solutions with the running cost worked out before the first line of code. If you have an idea and want to know what it would cost monthly, send us the use case.