Cloudflare cut its AI code review cost 68% with caching and routing

Cloudflare cut the cost of its in-house AI code reviewer by 68%. An average review at high effort now costs $0.77, down from $2.43. Reviews also got faster. The median time fell from 4 minutes 9 seconds to 1 minute 32 seconds. The company says it got there by reusing work the AI had already done and by choosing models more carefully.

Why the bill kept growing

In April, an average review cost $1.19. By the end of August it was about $2.46, with no big change to how the system was built. Most of the rise came from model changes. In early August every reviewer moved to Anthropic models, which tested better on this work. That raised the cost per job from $1.38 to $2.03. One reviewer moved to Claude Opus 4.7 and went from about $0.25 to $0.88 per review.

Reviews were also getting bigger. Tokens, the small chunks of text an AI reads and writes, more than tripled per review, from 0.91 million to 2.99 million.

Where the money went

Just before the cleanup, output made up about 23% of spend. Cache writes were 43% and cache reads 34%. Most of the money went on context the model had already seen or would see again.

Each AI call sends the whole conversation so far. The code quality reviewer made 16.6 calls per review on average. So cost grows with both the number of calls and the size of the conversation.

The caching problem

Prompt caching lets an AI provider reuse the start of a prompt it has already processed. Writing to the cache costs 1.25 times the normal input price. Reading costs 0.1 times. An entry expires after five minutes unless something reads it.

Every review session starts with the same 11k to 21k tokens of tool definitions and instructions. That should have been a cache hit nearly every time. Instead, the first request found a cached prefix only 14% to 26% of the time.

The cause was how the older OpenCode v1 built its prompt. It put the shared instructions and a section with the working directory and date into one block, with one cache marker at the end. The shared part was never stored alone, so one repository could not reuse what another had written. On September 16, Cloudflare split that block in two at a marker.

Other changes

About 40% of the savings came from code changes. The rest came from testing which models to use and how much effort they spend. The reviewer also moved to OpenCode v2, a full rewrite. In side-by-side testing, v1 and v2 agreed on the approval decision in 98% of merge requests. Full-tier reviews on v2 finish about 13% faster.

What it means for businesses

The lesson reaches beyond Cloudflare. AI running costs can climb quietly as models and tasks grow. Two levers helped most here: keeping the repeated part of a prompt identical so it can be cached, and matching the model and effort level to each job.