Skip to content
Antoine Le LouëtIndependent senior developer (Home)
Back to the Journal

Advice5 min read

Reduce token usage: 5 tips for using AI

Select useful context, make requests precise and limit retries: five practical tips for using fewer tokens with AI.

Instructions and useful excerpts form the context sent to the model. It produces a response. Subsequent exchanges may reuse previously transmitted context.
Conceptual diagram: usage covers transmitted context and generated tokens. Some models also use reasoning tokens that are not visible in the response.

A short response can hide a lot of context

You attach a long document, ask a question, then request several rewordings. On screen, the latest messages look short. Yet the model may receive part of the earlier exchange again. To reduce token usage, look at all the work needed to obtain a usable response.

A token is a unit of text processed by the model, not necessarily a complete word. Tokenisation depends on the model and text. Inputs include your instructions and the retained context; outputs include generated tokens. Some models add internal reasoning that may be counted even if the displayed answer is brief.

The following examples are fictional. Technical mechanisms are documented in the OpenAI sources at the end of the article; their exact behaviour depends on your tool. In a subscription application, the displayed quota does not necessarily correspond to per-token billing.

1. Send the excerpts that serve the question

Before attaching a file, ask what information the model needs to find in it. To explain an error, start with the complete message and relevant code section. Also say what triggers the problem. An entire repository or hundreds of unrelated log lines add context to process.

For a document, identify the section and preserve definitions needed to understand it. Remove duplicates and repeated headers. If using a development agent, ask for a targeted search before reading whole files. Keep enough context to avoid an answer based on missing information.

Fictional example of a targeted request
Explique pourquoi le filtre disparaît au passage à la page suivante.
Je joins la fonction qui construit ce lien et un exemple d’URL.
Commence par ces éléments. Si une information manque, précise
le fichier ou le comportement à vérifier avant de proposer un correctif.

2. Define the expected result in the first message

A request such as ‘improve this text’ opens many possibilities. Specify its audience and what it should help them do. Add constraints that actually change the result. I prefer spending a few lines on this information to multiplying corrections after an overly general first answer.

For example: ‘Rewrite this message for a client waiting for a delivery date. Keep the confirmed date and explain the next follow-up. Do not promise any additional deadline.’ If an essential choice remains open, ask the AI to flag it before drafting. A longer instruction may reduce back-and-forth; this still needs checking across the complete task.

3. Request a response sized for its use

If you need to compare two options, a short table may be enough. For a code fix, ask for the patch and relevant checks without copying every unchanged file. Specify a concrete length or format: ‘one paragraph of no more than 150 words’ is more precise than ‘be concise’.

Keep the explanations needed to decide whether the answer is reliable. An answer so short that it requires several follow-ups may cost more across the full exchange. In an API, an output limit set too low may also interrupt generation; with some reasoning models, it includes internal tokens. Requesting a brief answer therefore does not, on its own, guarantee low total usage.

4. Start again with a summary when the subject changes

A conversation mixing several subjects may retain history that is no longer useful. When moving to a distinct task, open a new exchange with a self-contained note. Include validated decisions and information still needed. A summary prepared or reviewed by you avoids transmitting assumptions that have become false.

If the new task still depends on details of the previous thread, stay in that thread or attach essential excerpts. Generating a summary also consumes tokens: the detour is mainly justified when it will be reused. In the OpenAI API, referencing a previous response does not make the history’s input tokens free.

In a thread retaining its full history, context A is reused in subsequent calls, then joined by B and C. For a distinct task, a new conversation can send only a useful note and the new request.
Fictional example: blocks represent context, not token counts. A new conversation retains only what you send it; check that the note contains the necessary information.

5. Scope tools and retries, then measure

With an agent, every additional search or attempt may trigger another model call and add results to the context. First request checks useful for the task. Prefer a relevant log excerpt to full output while retaining errors needed for diagnosis. In an API integration, exposed tool descriptions may also count towards input tokens.

Set a stopping criterion: ‘If the same test still fails after the fix, present the error and your next hypothesis before trying again.’ Avoid retrying an unsatisfactory answer without new evidence. For exact counting or filtering, an appropriate local tool can also avoid a model call.

To verify the effect of these habits, compare similar tasks and count all calls until the accepted result. With an API, record input and output tokens from usage data, as well as reasoning details when supplied. Also compare the quality obtained. A cheaper model can reduce the bill without reducing the number of tokens.

Context caching may change the cost of a reused input depending on the provider and its rules. It does not remove that text from the context. Do not lengthen a request solely to reach a caching threshold. On your next task, first try removing an unnecessary attachment or requesting the response format you actually need, then observe the follow-ups avoided.

Sources and useful links

Share on LinkedIn (new tab)

Keep reading
All articles in this category