How to save tokens when working with AI
Practical tips for reducing token usage and AI costs through model selection, planning, compaction and subagents.
The cost of AI depends on three things:
- Which model we use.
- How long the chat is (its context).
- Whether we use caching.
We will explain what each point means and how it works in practice. At the end, we will bring it all together with practical advice on how to approach your work and reduce costs.
1. The right model for the job
New AI models are released every week, whether by major American or Chinese research labs or by technology companies. Different models have different strengths and perform better or worse in different categories. These differences often only become apparent in use, as benchmarks struggle to reveal them. A simple rule of thumb, however, is that the larger the model, the more capable it is—but also the more expensive.
In Praktik, we essentially have three categories of models: small, medium and large. We use small models to summarise legislation, extract contracting parties, name chats and handle many other straightforward tasks. Medium models are used most often, whether in tables, by research subagents (more on those later) or in the chat itself. Large models are especially useful for longer tasks, planning or complex cases that require more reasoning.
2. Cost depends on context
Cost also depends on the context the agent works with. Think of context as the information the agent has loaded. When you send it a message or ask it to read a statute or court decision, all of this is added to its context. It is like a file to which more text keeps being added. That text can come from you, a document or a response the agent writes. The longer the context, the more tokens the model has to process and the more expensive the response becomes.
Context is, of course, limited. Most new models have a context window of 1 million tokens, with one token corresponding roughly to one syllable.
(Try seeing how text is split into tokens.)
The model can only work with a limited number of words when responding. Once the limit is exceeded, older information may no longer fit into its context. You can still see it in the chat, but the AI may not have access to it when answering.
This is why automatic compaction takes place. We take all the messages and create a summary, discarding unnecessary information and retaining only what matters. The indicator at the top of the chat shows how much of the context is being used. If the agent has spent a long time searching and has accumulated a lot of information, you can also trigger compaction manually with specific instructions to improve the summary.
Alongside the strategies we cover at the end, you can use subagents to manage context efficiently. Imagine giving a colleague a specific research task and receiving only the result. Only the task and the result are then loaded into the main agent’s context, rather than every step taken to reach that result.
3. Caching
Last but not least, there is caching. AI models do not work directly with text, but with numbers (more about prompt caching), so text must be converted into a numerical representation the AI can work with. This process is not free and accounts for a significant part of the cost of a response.
Because this conversion is the same each time, it does not have to be repeated: we can store it in a cache. Caching makes context processing faster and many times cheaper. However, the cache is cleared after a certain period. So even if you have a long chat, actively working in it costs substantially less than sending one message a day.
The topic itself is fairly complex and depends on many details, but you do not need to explore it in depth. That is our job.
Practical advice for using AI
Choose a model to suit the task
In Praktik, you can choose from three model tiers. For everyday work, we recommend the “normal” model, as it is designed for that purpose. We recommend the “advanced” model in specific cases, namely when:
- The answer requires complex reasoning because there is no clear answer or obvious route to it.
- The task involves many steps and requires a complex procedure. One example is editing a complex document while following a specific process.
- You need to create a plan to tackle the task more effectively, as a more advanced model is better at planning and identifying gaps in the plan.
The advanced model does not have a larger context window. Its advantage is better reasoning and planning for demanding tasks.
Create a plan
We recommend using the advanced model for planning. The benefit of a plan is that you and the agent agree on the entire task in advance. You know how you want to proceed and what to search for, so the agent does not have to invent an approach as it goes. For example, you can say: “First, propose a process for reviewing this contract, and wait for my confirmation before starting.”
If you are reviewing a lengthy contract, for example, give the agent the objective, source materials and approved plan together, and let it complete the review in one continuous run. Caching makes the work substantially cheaper.
Another benefit of planning with the agent is that it avoids getting stuck or losing track of the task. As it works, it will refer back to what it needs to do and how to proceed, updating the plan with new information and recording the parts it has completed.
Stop when necessary
It is just as important to tell the agent when to stop. If you ask it to find something, it will keep looking. If the answer is not in legislation, court decisions, opinions or other ministry documents, it will start searching the internet for an answer to your question. On the one hand, this is useful. But if you know that the information may not exist, or you know which sources the agent should consult, it is worth saying so. This makes the response faster and cheaper.
We also recommend asking the agent to get your confirmation before it starts work. For example, before reviewing a contract, have it explain which areas it intends to change and why. You can easily review and adjust its approach.
Use subagents for specific tasks
We mentioned subagents as a way to delegate work and receive only the result. The key is that the main agent does not need to “remember” all the steps taken to reach that result.
For example, suppose you need to verify company details in a contract against the Slovak Commercial Register (ORSR), check every reference to the Slovak Commercial Code, and read the court decisions mentioned in the document to verify that they exist and are cited correctly. You can simply add this to the end of your prompt: “Use a subagent for each task.” That is all it takes to have those tasks carried out by subagents. They are not only cheaper but also faster, as the individual tasks can run in parallel.
When to compact
It is not always obvious when compaction is worthwhile. There is certainly no need to overdo it or use it too frequently, but it is one way to reduce costs and improve response quality.
A good example is researching a topic you are unfamiliar with, where you cannot give the agent precise direction. It has to do a lot of searching, reading and analysis. Even after it prepares an answer, it has loaded a great deal of unnecessary information that does not help your work. If you want to take the result and continue working with it, you no longer need that extra information.
In this case, we recommend manually compacting the conversation, ideally with a specific prompt focused on the purpose of the summary.
In other words, if the agent has information that is unnecessary for its current task, it is better to compact.
You do not need AI for everything
New AI technology is remarkable because it can help in almost every aspect of legal practice. That does not mean it is the best option for everything: specialised workflows can be not only much cheaper but also considerably better.
If you need to translate or anonymise a document, or verify references within it, we do not recommend using the chat for that. For translations, we have a dedicated DeepL-based feature in documents and in Word. It is faster, cheaper and, most importantly, produces better results.
The same applies to anonymisation and checking the validity of references. Praktik can handle those better in Word.
Write down your workflows
One of the biggest problems when using AI is that the system does not know how to approach your task. It may eventually reach an answer, but it takes a long time and tries approaches that do not lead to the right result.
If you have a task that comes up frequently, we recommend turning it into a skill (video on editing and creating skills). The agent will then load the workflow for that task and know exactly what to do and how to proceed.