Software usually costs once to build and then runs at a roughly fixed cost. AI features are different: the running cost scales with usage. The more successful the feature, the larger the bill. Not anticipating that is a common reason a perfectly good feature gets switched off after three months.
How billing works: tokens in and tokens out
Model providers bill by token, a unit smaller than a word. In English a token averages around three quarters of a word, so a page of roughly 500 words lands near 650–700 tokens; Vietnamese runs higher per page because of its syllable structure.
The two kinds are billed separately and at different rates:
- Input tokens — everything you send: the question, the system instructions, the retrieved passages, the conversation history.
- Output tokens — what the model writes back. Usually several times the price of input.
Actual prices change constantly and differ between models, so do not copy a figure into a long-range financial plan — look up the rate card when you build. What does not change is the structure of the cost, and that is what has to be designed properly.
Where the cost quietly balloons
In the systems we have reviewed, budget overruns almost always come from input tokens, not output:
| Cause | Why it costs | What to do |
|---|---|---|
| Sending the whole conversation every turn | By turn 20, each question carries the previous 19 | Summarise the older part, keep only the last few turns verbatim |
| Stuffing in too many retrieved passages | Fetching 20 to be safe instead of the right 4 | Improve retrieval, re-rank, then cut |
| A long-winded system instruction | A 2,000-token block attached to every question, all day | Trim it, and use context caching if the provider offers it |
| Using the strongest model for everything | Classifying a short question does not need the priciest model | Route by difficulty: small model for easy work, escalate only when needed |
How to estimate before you build
The minimum formula to present to management:
Monthly cost ≈ uses per month × (input tokens per use × input price + output tokens per use × output price).
What matters is not the final figure but the three scenarios: current usage, usage if the whole company adopts it, and usage if you open it to external customers. The third is often dozens of times the first, and that is when limits become necessary.
Seven ways to cut cost without cutting quality
- 1Cache repeated answers. A great many questions are word-for-word identical; returning a stored result is free.
- 2Route by difficulty. Most usage is easy work that does not need the most expensive model.
- 3Trim the context. Four correct passages beat twenty gathered for safety — cheaper and more accurate at once.
- 4Use context caching for the unchanging system instruction, where the provider supports it.
- 5Batch the work that needs no immediate answer, such as classifying documents overnight.
- 6Cap the answer length. Three paragraphs are not needed when one sentence will do.
- 7Measure before optimising. Log token counts per request type; there is almost always one type carrying most of the bill.
Compare against what you already spend
In the end the comparison is not "is AI expensive" but "how does it compare with the current way". If a feature costs a few million dong a month and replaces 25–30 hours of data entry, the arithmetic is obvious. If it costs a few million a month to provide a convenience somebody occasionally uses, that is equally obvious in the other direction.
Want to talk specifics?
SealCore surveys at your premises and sends a fixed quote after the first session — including when the conclusion is that you do not need custom software.
