A few months ago I became convinced my AI assistant was bleeding money through incompetence. It felt true. Sessions ran long, things failed, and the meter was always spinning. So I did what I do with any suspected leak: I measured it. I streamed 45 days of local session transcripts through a script that classifies every failed tool call and prices the recovery turn that follows it. It covered 660 session files, roughly June through early August. The parse ran entirely on my machine and cost zero tokens, which turns out to matter more than I expected, because once something is free to measure you actually measure it weekly, and I had been treating measurement itself as a one-time project.
The number I went looking for was big. Here is what the audit actually found: 729 failed tool calls across 45 days, about sixteen a day, and 718 recovery turns where the model stopped to clean up after a failure. Those recovery turns cost roughly 15.85 million of my local “effective unit” weighting. Total spend in the same window was about 1.05 billion units, which puts the cleanup bill at one and a half percent.
The mistakes were real, and fixing them is worth doing, but they were not where the money went.
About forty percent of the failures were genuine agent mistakes: edits against stale files, paths that stopped existing two sessions ago, a PowerShell script using an operator the installed version doesn’t have, tool-call JSON the schema rejected. The other sixty percent were the environment being the environment. There were pages that never reach DOM-idle, so the browser tool waits 45 seconds and gives up. Screenshot injection timed out on busy screens. Windows locked files underneath edits at the worst moments. The worst offender is a professional networking site you have heard of. It never idles, on purpose, and the tool waits faithfully every time. None of that is fixable by prompting harder, and I had been mentally invoicing all of it to the model.
While I had the transcripts open, I priced the thing nobody warns you about. A long session re-reads its own history on every reply, and the meter spins for every one of those re-reads. This is the quiet failure mode of AI-assisted work: nothing errors, nothing alerts, the session just grows and every turn gets a little more expensive until a routine question is quietly hauling a few megabytes of dead conversation uphill with it. The audit put that re-read tax at roughly 80% of total spend. The single worst session had burned 240 million cache-read tokens on its own, across 665 messages and a 7.2MB transcript, and I couldn’t have told you what most of those messages were about.
Three sessions like that accounted for about 493 million tokens in one week. That was 20% of the week’s entire burn, sitting in three tabs I kept open because closing them felt like losing something.
I made three changes.
The first one was the least glamorous. The hygiene rule stopped being aspirational. The top burner sessions get archived the moment the audit names them, and a fresh session opens with a one-line handoff so nothing of value depends on the old context surviving. It sounds too small to matter, and that is exactly why it took an audit to make me actually do it.
Then I checked the boring tiering instead of assuming it. Every scheduled routine in the estate is pinned to the cheapest model that can do the job, and the model-fit check confirmed that held, with one catch worth naming. The same check flags workflow fan-out calls that inherit the premium model by omission, which is exactly the kind of leak you only see when you go looking. When it flags one, pinning a cheaper tier is a small config change, so there is no reason to leave it inherited.
The last change was making the measurement itself a routine. The scanner is about 200 lines of PowerShell, it runs Sunday, it parses the week’s transcripts locally and writes a snapshot file, and because it never calls the API it never shows up in its own report, which keeps the measurement honest in the cheapest possible way.
Here are the snapshots, for the record. Rolling seven-day cache-read volume across all models: 2.45 billion tokens on August 9. 447 million on August 27. 151 million on August 30. Weekly message count fell from 17,385 to 2,899 to 1,014 over the same snapshots. The two later windows overlap and my estate migration landed in the same stretch, so I won’t pretend I can cleanly attribute the drop to any single cause. The message count is the honest signal, because closing sessions is a behavior, and behavior shows up in the count no matter what the platform charges per token.
One caveat before anyone quotes these numbers at a vendor: “effective units” are my local weighting for comparing spend across token types, and the vendor’s invoice uses different math. The ratios are the finding, not the absolutes, and the ratio says my own habits were the line item, which was not the conclusion I wanted from a report I built to blame the model.
The agent was never the expensive part. I was the expensive part, one open tab at a time.