Blog

Verbosity Index, Edition 2

Same six prompts, same method, 72 fresh metered runs across six models including the new Fable 5.1. This edition also corrects a confound in Edition 1: one of the models we compared was not counting tokens the same way as the others.

By the Unbiased Team · published August 25, 2026 · 72 runs metered August 25, 2026; rates verified the same day

Edition 2 re-measured output volume across six Claude models on identical prompts (72 metered runs). Claude Opus 5 has the highest token index at 1.62 against Opus 4.8's 0.89, so it emits 82% more billed output for identical work on an identical rate card. Edition 2 also measures output characters, which removes a tokenizer difference that inflated Edition 1's cross-generation comparisons.

What the index measures

Two models on the same rate card can bill very differently, because price is per token and models do not write the same number of tokens for the same task. The Verbosity Index measures that: six task types, two runs per model, identical prompts, no length guidance, default settings. Each model's score is the geometric mean of its per-task output volume relative to the median across all models measured. 1.00 is the median. Higher costs you more for identical work.

Edition 1 measured tokens only. Edition 2 measures tokens and characters, and publishing both is the point of this edition.

The confound in Edition 1

Anthropic's pricing documentation notes that models from Claude 4.7 onward use a newer tokenizer which "produces approximately 30% more tokens for the same text." That single sentence undermines any token-count comparison that spans the two generations, and Edition 1 spanned it: five of its six models used the newer tokenizer and Haiku 4.5 used the previous one.

So we measured it rather than taking the 30% on faith. Across the same battery, models on the newer tokenizer averaged 2.172 characters per output token; Haiku 4.5, on the previous tokenizer, averaged 3.409. That is 57.0% more text carried per billed token, which is nearly double the documented figure rather than close to it.

Two honest caveats on that number. The documentation says the exact increase depends on content and workload shape, and our battery is deliberately code and JSON heavy, which is punctuation dense and tokenizes worse under the newer scheme; a prose-only battery would likely land closer to the stated 30%. And characters per token is a property of the text as well as the tokenizer, so a model that writes tighter prose shifts the ratio too. We are reporting the measurement, not claiming to have isolated the tokenizer perfectly.

What this means concretely: part of Haiku's flattering Edition 1 score was not concision, it was a tokenizer that charges fewer tokens for the same prose. Within a tokenizer generation the token index is a fair comparison. Across generations, only the character index is.

Edition 2 results

On tokens, Opus 5 remains the outlier at 1.62 against Opus 4.8's 0.89: 82% more billed output for identical prompts on an identical $5/$25 rate card. On characters the same gap is 58%, so this one is real verbosity rather than tokenizer artifact. Opus 5 genuinely writes more.

Fable 5.1, measured here for the first time, scores 0.86 on tokens against Fable 5's 1.01 on the same $10/$50 card. The six-task battery billed $0.2904 on 5.1 and $0.2556 on 5.

Haiku 4.5 remains the cheapest per task by a wide margin ($0.0219 for the battery), and that conclusion does not depend on the tokenizer question at all, because its rate card is five to ten times lower than everything above it. The tokenizer correction changes how we compare its concision, not whether it is cheap.

What to do with this

If you are choosing between two models on the same rate card, the token index is the number that predicts your bill, and the gap can be larger than any discount you negotiate. If you are comparing across model generations, use the character index or measure your own tasks, because token counts are not commensurable.

And treat any model swap as a billing change even when the price sheet does not move. That is the finding both editions keep producing: the rate card is not the bill.

Reproduce this

The six prompts are published verbatim in our battery script, the per-run token and character counts are in the data file below, and the method is two runs per task per model at default settings with no length guidance. Nothing here needs our word for it.

Data: verbosity-index.json. Script: verbosity-battery.py. Edition 1, for comparison: the original index.

Questions, answered