How to reduce token usage in an AI agent

Fixing the option to disable bot memory

The original bot.py trimmed history with [-2 * HISTORY_TURNS:]. At HISTORY_TURNS=3, it kept the last three question-and-answer pairs. At 0, the expression became [-0:] and kept the entire list. The option intended to disable memory did not work correctly.

We asked the model to move this behavior into keep_history(history, turns). The contract was explicit: zero returns an empty list, a negative value raises ValueError, and a positive value keeps the most recent pairs. The original list must stay unchanged.

This task has a short behavior check, making it possible to compare costs and verify that both context variants produced a suitable solution.

Snippet supplied in the focused variant

HISTORY_TURNS = 3
history = context.user_data.get("history", [])
context.user_data["history"] = (
    history + [user_message, {"role": "assistant", "content": reply}]
)[-2 * HISTORY_TURNS:]

Telegram bot tutorial with the corrected code

Comparison settings

The test ran on October 23, 2026, through /v1/chat/completions using claude-haiku-4.5. Both variants used the same system instruction and task, temperature=0, max_tokens=400, and disabled automatic SDK retries.

The full variant appended the entire original bot.py to the task. The focused variant included only HISTORY_TURNS and the history-trimming code. The original source file and comparison script are provided below.

The six requests ran in this order: full, focused, focused, full, full, focused. Each started a new conversation. The requests did not include agent tools for reading files or running commands; separate Python code checked the model's answers.

The measured task and source file contain Russian text. The prompt block and executable samples retain those strings so the recorded experiment can be reproduced. The function contract is explained in English above. Translating a prompt changes the input and may change token counts.

Identical task for both variants (original Russian prompt)

В Python-боте история ограничивается выражением
(history + [user_message, assistant_message])[-2 * HISTORY_TURNS:].
При HISTORY_TURNS=0 оно сохраняет всю историю вместо отключения памяти.
Напиши функцию keep_history(history, turns), которая исправляет это поведение.
history содержит полный список чередующихся сообщений user/assistant, только полные пары.
turns - целое число. При turns=0 верни пустой список; при turns<0 подними ValueError.
При turns>0 верни последние turns полных пар или все сообщения, если их меньше.
Всегда возвращай новый список и не меняй исходный.
Верни только одну функцию keep_history на Python без Markdown, импортов и объяснений.

Token counts and billed costs for six requests

All three requests within each group returned the same input and output token counts. The full file used 1,756 input tokens and 59 output tokens; the snippet used 359 input tokens and 64 output tokens. The cached_tokens field was 0 in all six responses.

The table uses costs from VOID's request history. At the time of the test, the price was $0.57 per million input tokens and $2.20 per million output tokens.

All six functions passed eight behavior checks. We removed Markdown code fences from the responses before testing, without editing the Python code. The focused-context solution made an unnecessary list copy but met the behavior requirements.

In this test, lower input usage accounted for the saving. The model did not produce a shorter answer: output grew from 59 to 64 tokens in the focused variant. A different fix, model, or history may give different results.

Measured results

Variant    Requests  Input/request  Output/request  Cost/request, USD
Full file  3         1756           59              $0.00113072
Snippet    3         359            64              $0.00034543

Three full-file requests: $0.00339216
Three snippet requests:   $0.00103629

Solution from the full-file request

def keep_history(history, turns):
    if turns < 0:
        raise ValueError("turns must be non-negative")
    if turns == 0:
        return []
    return history[-2 * turns:]

Solution from the focused-context request

def keep_history(history, turns):
    if turns < 0:
        raise ValueError("turns must be non-negative")
    if turns == 0:
        return []
    return (history + [])[-(2 * turns):]

How we checked the solutions

With three pairs in the input history, we tested keeping zero, one, two, three, and nine pairs. Each case had to return a new list without modifying the original. Two more checks used empty history; the last expected ValueError for a negative pair count.

To check a generated function yourself, inspect its code and save it in solution.py without Markdown fences. Save the code below as test_history.py, then run python test_history.py. These checks assume the stated contract: inputs contain complete user/assistant pairs.

test_history.py (original test messages)

from solution import keep_history

history = [
    {"role": role, "content": str(turn)}
    for turn in range(3)
    for role in ("user", "assistant")
]
original = list(history)
for turns, expected in (
    (0, []),
    (1, history[-2:]),
    (2, history[-4:]),
    (3, history),
    (9, history),
):
    result = keep_history(history, turns)
    assert result == expected
    assert result is not history
    assert history == original

assert keep_history([], 0) == []
assert keep_history([], 3) == []
try:
    keep_history(history, -1)
except ValueError:
    pass
else:
    raise AssertionError("Отрицательное число пар должно вызывать ValueError")

print("8 проверок поведения пройдены")

Apply the result to your own agent

For a local code change, first find the relevant function and supply it with the requirements. Include callers, types, or configuration when the issue depends on them. In our task, Telegram handling, client setup, and quota storage did not affect keep_history.

Keep a long log in a separate file and pass the model the failed test name, failure location, and diagnostic message. If the reduced context forces another read of the same log, include those extra calls in the cost comparison.

Preserve tool calls together with their results. Trimming a message list by item count can leave a tool response without its matching call. Use context reduction that understands the message structure for these conversations.

Run the same result checks after changing the context. A cheaper incorrect solution does not establish that the task costs less to finish.

Repeat the comparison with a script

Use the Python environment and API key from the bot tutorial. This script needs openai==3.37.0 and python-dotenv==1.2.4. Set VOID_API_KEY and VOID_MODEL=claude-haiku-4.5 in .env. It does not use a Telegram token.

Save the original bot source below as bot-baseline.py and the comparison script as compare_context.py. Run python compare_context.py bot-baseline.py. The script reads the source as text and sends six paid model requests; it does not execute the bot file.

After each request, answers and usage are saved in comparison.json. The syntax_ok field checks syntax only. Use the code in the previous section to check behavior. If any request fails, the script stops without an automatic retry.

Repeated runs can differ because of model versions, file formatting, or cache state. For your own tasks, compare the full cost of reaching an accepted result, including failed attempts and repeated reads.

compare_context.py (original measured prompts)

import ast
from datetime import datetime, timezone
import json
import os
from pathlib import Path
import sys

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv(Path(__file__).resolve().parent / ".env")
model = os.environ["VOID_MODEL"]
task = """В Python-боте история ограничивается выражением
(history + [user_message, assistant_message])[-2 * HISTORY_TURNS:].
При HISTORY_TURNS=0 оно сохраняет всю историю вместо отключения памяти.
Напиши функцию keep_history(history, turns), которая исправляет это поведение.
history содержит полный список чередующихся сообщений user/assistant, только полные пары.
turns - целое число. При turns=0 верни пустой список; при turns<0 подними ValueError.
При turns>0 верни последние turns полных пар или все сообщения, если их меньше.
Всегда возвращай новый список и не меняй исходный.
Верни только одну функцию keep_history на Python без Markdown, импортов и объяснений."""
short_context = """HISTORY_TURNS = 3
history = context.user_data.get("history", [])
context.user_data["history"] = (
    history + [user_message, {"role": "assistant", "content": reply}]
)[-2 * HISTORY_TURNS:]"""

if len(sys.argv) != 2:
    raise SystemExit("Запуск: python compare_context.py bot.py")

full_context = Path(sys.argv[1]).read_text()
report = {"model": model, "date": datetime.now(timezone.utc).isoformat(), "runs": []}
output = Path(__file__).resolve().parent / "comparison.json"
with OpenAI(
    api_key=os.environ["VOID_API_KEY"],
    base_url="https://void-api.tech/v1",
    timeout=75,
    max_retries=0,
) as client:
    for variant in ("full", "focused", "focused", "full", "full", "focused"):
        context = full_context if variant == "full" else short_context
        response = client.chat.completions.create(
            model=model,
            messages=[
                {"role": "system", "content": "Ты исправляешь код Python. Соблюдай заданный контракт функции."},
                {"role": "user", "content": task + "\n\nИсходный код:\n" + context},
            ],
            temperature=0,
            max_tokens=400,
        )
        code = response.choices[0].message.content or ""
        if code.startswith("```python\n") and code.rstrip().endswith("```"):
            code = code.strip()[len("```python\n"):-3].strip()
        try:
            ast.parse(code)
            syntax_ok = True
        except SyntaxError:
            syntax_ok = False
        row = {
            "variant": variant,
            "response_id": response.id,
            "usage": response.usage.model_dump() if response.usage else None,
            "syntax_ok": syntax_ok,
            "answer": code,
        }
        report["runs"].append(row)
        output.write_text(json.dumps(report, ensure_ascii=False, indent=2))
        print(json.dumps({"variant": variant, "usage": row["usage"], "syntax_ok": syntax_ok}, ensure_ascii=False))

print(f"Ответы и токены сохранены в {output.name}. Проверьте поведение функций отдельно от синтаксиса.")

Set up Python and your API key

View actual billed costs

Original source file used in the test

This is the initial file supplied to the model in the full-context variant. It includes the HISTORY_TURNS=0 bug because that code is part of the test input. Use the corrected version in the bot tutorial for your own bot.

Save this block as bot-baseline.py. You do not need to execute it to compare token usage.

bot-baseline.py: original experiment input

import logging
import os
import sqlite3
from datetime import datetime, timezone
from pathlib import Path

from dotenv import load_dotenv
from openai import APIError, AsyncOpenAI
from telegram import Update
from telegram.ext import Application, CommandHandler, MessageHandler, filters

ROOT = Path(__file__).resolve().parent
load_dotenv(ROOT / ".env")
OWNER_ID = int(os.getenv("TELEGRAM_ALLOWED_USER_ID", "0"))
MODEL = os.environ["VOID_MODEL"]
DAILY_REQUESTS = 20
HISTORY_TURNS = 3
MAX_INPUT_CHARS = 3000
MAX_OUTPUT_TOKENS = 600

logging.basicConfig(level=logging.WARNING)
client = AsyncOpenAI(
    api_key=os.environ["VOID_API_KEY"],
    base_url="https://void-api.tech/v1",
    timeout=60.0,
    max_retries=0,
)
db = sqlite3.connect(ROOT / "quota.sqlite3")
db.execute("CREATE TABLE IF NOT EXISTS quota (day TEXT PRIMARY KEY, used INTEGER NOT NULL)")
db.commit()


def reserve_request():
    day = datetime.now(timezone.utc).date().isoformat()
    with db:
        cursor = db.execute(
            "INSERT INTO quota(day, used) VALUES (?, 1) "
            "ON CONFLICT(day) DO UPDATE SET used = used + 1 WHERE used < ?",
            (day, DAILY_REQUESTS),
        )
        return cursor.rowcount == 1


async def start(update, context):
    await update.message.reply_text(
        f"Ваш Telegram ID: {update.effective_user.id}.\n"
        "Укажите его в TELEGRAM_ALLOWED_USER_ID и перезапустите бота.\n"
        "Команда /reset очищает историю диалога."
    )


async def reset(update, context):
    if update.effective_user.id != OWNER_ID:
        return
    context.user_data.clear()
    await update.message.reply_text("История очищена. Дневной счётчик сохранён.")


async def answer(update, context):
    if update.effective_user.id != OWNER_ID:
        await update.message.reply_text("Это личный бот. Доступ к модели закрыт.")
        return
    text = update.message.text.strip()
    if not text or len(text) > MAX_INPUT_CHARS:
        await update.message.reply_text(f"Отправьте от 1 до {MAX_INPUT_CHARS} символов.")
        return
    if not reserve_request():
        await update.message.reply_text("Дневной лимит исчерпан. Обновление в 00:00 UTC.")
        return

    history = context.user_data.get("history", [])
    user_message = {"role": "user", "content": text}
    messages = [
        {"role": "system", "content": "Отвечай по-русски, кратко и по существу."},
        *history,
        user_message,
    ]
    try:
        response = await client.chat.completions.create(
            model=MODEL,
            messages=messages,
            max_tokens=MAX_OUTPUT_TOKENS,
        )
    except APIError as error:
        logging.warning(
            "Void API: %s, HTTP %s", type(error).__name__, getattr(error, "status_code", None)
        )
        await update.message.reply_text(
            "Не удалось получить ответ. Проверьте ключ, баланс и модель в VOID. "
            "Попытка учтена в дневном лимите."
        )
        return

    choice = response.choices[0] if response.choices else None
    reply = (choice.message.content or "").strip() if choice else ""
    if not reply:
        await update.message.reply_text("Модель не вернула текст. Попытка учтена в лимите.")
        return
    context.user_data["history"] = (
        history + [user_message, {"role": "assistant", "content": reply}]
    )[-2 * HISTORY_TURNS:]

    for offset in range(0, len(reply), 2000):
        await update.message.reply_text(reply[offset : offset + 2000])
    if choice.finish_reason == "length":
        await update.message.reply_text("Ответ остановлен по лимиту токенов.")
    if response.usage:
        print(
            f"model={response.model} input={response.usage.prompt_tokens} "
            f"output={response.usage.completion_tokens}",
            flush=True,
        )


async def on_error(update, context):
    logging.warning("Ошибка обработчика: %s", type(context.error).__name__)


async def shutdown(application):
    await client.close()
    db.close()


def build_app():
    app = (
        Application.builder()
        .token(os.environ["TELEGRAM_BOT_TOKEN"])
        .concurrent_updates(False)
        .post_shutdown(shutdown)
        .build()
    )
    private = filters.ChatType.PRIVATE
    app.add_handler(CommandHandler("start", start, filters=private))
    app.add_handler(CommandHandler("reset", reset, filters=private))
    app.add_handler(MessageHandler(private & filters.TEXT & ~filters.COMMAND, answer))
    app.add_error_handler(on_error)
    return app


if __name__ == "__main__":
    build_app().run_polling(allowed_updates=[Update.MESSAGE], drop_pending_updates=True)

Corrected bot code

What this example can tell you

We tested a small function with an explicit contract. This result does not establish the savings for a large refactor, a bug spanning several files, or a task involving web search.

When changing models, compare the cost of several completed tasks and the number of failures. When using a cache, check the actual cache fields and their prices: cached_tokens may already be included in the input total and must not be counted twice.

For your first agent comparison, choose a repeatable task and record the starting files and acceptance criteria. Then change one factor, such as the amount of code supplied, and compare costs for accepted solutions.

Models and current prices

VOID API reference

Configure retries in the OpenAI SDK