Claude Sonnet 5.5 task cost: why 30% less and 50% more describe different tests

Imagine a team deciding whether to move its queue of code fixes from Claude Sonnet 5 to Sonnet 5.5. Anthropic says the new model can cost up to 30% less per task; Artificial Analysis (AA) measured roughly 50% more at max effort on its Intelligence Index, using a pre-release deployment. Both observations can be true because the tasks and settings differ. For that team, the purchasing question is neither percentage: it is what the cheapest passing configuration costs per fix that survives tests and review.

This is not an accusation that one evaluator got its arithmetic wrong. It is a warning about a unit that sounds stable but isn’t. A “task” in a vendor coding test, a task in an index spanning multiple evaluations, and a ticket in your engineering backlog do not share an acceptance rule.

Two claims about “cost per task,” two different jobs

Anthropic’s launch announcement says Sonnet 5.5 “costs up to 30% less per task than its predecessor.” The predecessor is Sonnet 5; up to describes favorable results in Anthropic’s tests, not an across-the-board reduction in the token price or your next invoice. The announcement gives individual comparisons on coding and knowledge-work evaluations at particular effort settings. In some, Sonnet 5.5 at low or medium effort surpasses Sonnet 5’s best reported score for a fraction of its task cost. That is a useful clue about where to test first. It is not a service-level promise about every task in the queue.

AA asked a different question. Its Sonnet 5.5 report ran all five effort levels across the Intelligence Index, a blend of evaluations rather than a team’s actual tickets. At max effort, it reports $7.60 per Index task, about 50% more than Sonnet 5 on its cost-per-task measure. Sonnet 5.5 used about 193,000 output tokens per Index task at that setting. It also scored 56 on the Index, 18 points above Sonnet 5. The higher cost came with a different measured quality result; this is not a comparison at a fixed pass rate.

The chart below is AA’s original. Its axes belong to AA’s evaluation, not to Anthropic’s task selection. For a quick read on a phone, the comparison that matters is also in text:

Those figures are not two readings of the same instrument. Subtracting their percentages, or working backward from AA’s rounded “roughly 50%” to publish a precise Sonnet 5 dollar cost, would manufacture precision the sources do not provide. The defensible observation is that the workload and operating point changed. Neither public headline isolates how much of the difference came from task mix, effort, cache behavior, or the pre-release version.

The price sheet stayed level; the amount of work did not

Here is the part that makes the contradiction feel real to an API buyer. On Anthropic’s model pricing table, Sonnet 5 and 5.5 share listed rates for ordinary input ($2 per million tokens), output ($10), and cache reads ($0.20). For Sonnet 5.5, a prompt-cache write retained for five minutes costs $2.50 per million tokens; the one-hour write costs $4. Reads, writes, ordinary input, and output are separate billing lines. These are first-party standard rates; another platform’s contract, regional routing, or service tier may change the effective invoice.

Suppose the hypothetical team’s acceptance rule is “the patch passes tests and a reviewer accepts the change.” One model might spend more output tokens on its first attempt but pass. Another might write a short answer, fail a test, reopen the code, and need another turn. The price sheet cannot choose between them: it names the cost of tokens, while the team buys an accepted fix. The same is true in reverse. A higher-effort run can pay for more thinking than the task needs, so that a lower-effort run passes the same gate at lower cost. No outcome in this example is a measured result for either model; it describes the decision the team needs to instrument.

AA’s 193,000 output tokens per Index task at max makes the distinction visible. That number aggregates what its test consumed on one benchmark task; it is not 193,000 words in a single visible reply or the expected usage of your API request. The model can expend tokens on reasoning and tool-related output that never appears as the final answer. A fixed output price can therefore produce a higher task bill without any token price increase.

Caching can move it again. A prefix that hits a cache is billed at the read rate, whereas a new cached prefix incurs a write; you must actually enable and maintain useful caching in the request path. A long multi-turn agent does not automatically inherit AA’s cache-hit pattern. Record ordinary input, cache creations, cache reads, and output separately on every attempt. The first-party pricing page includes the full 1-million-token context for these models at standard per-token rates—no separate first-party long-context rate cliff—yet a larger prompt still contains more billable tokens. “No surcharge threshold” is not “a long session costs the same.”

Effort is an operating point, not a unit of quality

AA deliberately measured max. Anthropic’s Sonnet 5.5 announcement says Claude Code and the Claude apps default to medium effort, while the Claude Platform defaults to high. A developer looking at an app session, an API call with its effort omitted, and AA’s max-effort run may be looking at three different operating points. The $7.60 Index-task figure is not the price of a default request on any of those surfaces.

Even matching effort names between Sonnet generations is not a controlled experiment. Anthropic’s effort guide says Sonnet 5.5’s low, medium, high, xhigh, and max settings were recalibrated from Sonnet 5. Effort governs how much work the model puts into a response—including thinking and tool calls—but is not a fixed token budget. If a team migrates a setting called high unchanged, it has held the label constant, not necessarily the amount of thinking or the quality.

An engineering team should instead locate each model’s least expensive setting that clears its own acceptance bar. The settings need not have the same name. Comparing max with max is useful for a capability ceiling; it is not automatically the setting you would deploy for well-specified fixes. Equally, a cheap low-effort run that repeatedly fails is not evidence of savings. The grader needs to decide before anyone sees the bills.

The pre-release caveat changes the confidence, not the measured number

There is a second reason not to transplant AA’s result directly into a launch-day budget. AA says its evaluation used a pre-release deployment with a bug affecting structured-output requests—work that asks for a defined response shape, such as schema-valid fields. It says Anthropic fixed the bug for the public release and that AA will rerun relevant evaluations. Anthropic’s launch footnote also flags pre-release structured-output issues for GDPval-AA and AA-Briefcase, two knowledge-work evaluations, and expects little score change or a slight understatement.

As of our October 2 reread, AA’s article still does not report completed public-release rerun results. The fix is a fact about the public version; a revised Index score or cost is not yet an observation. It would be wrong both to discard the $7.60 measurement and to present it as a verified result for the fixed release. A fix might change task success or generated tokens, but neither source has measured the resulting change to that figure. AA also reports default safeguard fallback in about 0.1% of Index tasks, to Sonnet 5; record the actual route if that matters to your own workload, rather than claiming the entire Index came from the older model.

For the hypothetical code-fix team, structured-output cases deserve their own slice if the agent must return a strict schema. They should not be blended with plain-text fixes, then used to declare a general winner from a single average.

The test that can settle your choice

Take a small but representative backlog of tasks for which the team already knows what “done” means. Separate routine fixes from hard ones, and structured-output work from free-form work. Fix the prompts, tool access, time limits, and acceptance rubric before comparing models. Run the same tasks more than once when stochastic variation matters, with the same evaluation harness and caching policy. This costs real API money, so set a test budget rather than turning a blog’s suggestion into an uncontrolled production experiment.

For each model, start at an effort level you could plausibly ship, step down when it passes comfortably, and step up when it fails; use max only if the task warrants it. Keep the service tier and cache retention comparable. On every attempt, record the four billed token categories, retries, pass/fail, and elapsed time. Attribute failed attempts to the original ticket. Report both completion rate and total dollars spent per accepted ticket, by task slice; if a model cannot finish a slice within the allowed retries, do not hide those jobs by showing only the average cost of its rare successes. Keep latency and human review effort alongside—not inside a made-up API-dollar figure.

Then make a conditional decision. If Sonnet 5.5 meets the team’s gate at a less costly effort point, its cheaper completed work matters more than AA’s max-effort Index bill. If it needs expensive retries or max effort to pass, the vendor’s “up to 30%” result offers no discount to that queue. Revisit the judgment when AA publishes a public-release rerun, but do not wait for a general benchmark to tell you whether your own tests pass.