What makes a token expensive, and why it is not the model's size

What makes a token expensive, and why it is not the model's size

We keep a board of 101 AI models with a structured price field. It was built one model at a time, each figure read off the vendor's own pricing page on the day that model was listed. Nothing in it is estimated and no figure was copied from another table.

Read across that column, it contradicts the most common explanation of inference cost in circulation. The story being told everywhere this month is about size: models are getting smaller, therefore tokens are getting cheaper. The engineering behind that story is real — Cactus Compute has put a working automation model into 8 to 29 megabytes, and PrismML has compressed a twenty-seven-billion-parameter model into a ternary format. We added both to the board this week.

Neither changes what a buyer is charged per token, because a token's price is decided mostly by three things, and none of them is how big the model is.

Here is the whole of it, counted from our own board on 19 September 2026.

What the board can and cannot price

The board holds 101 models. Of those, 44 carry a structured specification — provider, licence, context window, pricing — and 23 publish a per-token price we could record. 78 publish no price at all. Fifty-seven of those 78 are older rows that pre-date the specification field entirely and carry no structured data; the other twenty-one have a specification but no price in it.

That split matters before any of the numbers below, so it goes first. Three quarters of this board cannot be priced, and the reason is not carelessness. It is that a large share of recent releases have no per-token price by construction. Granite 4.2 at 8B and 30B, Qwen3.8-27B, GLM-5.3-Flash, Kimi K3, Atria Dawn Preview, Bonsai 2 27B, Needle 3 and LimiX-2 all publish weights instead of a rate card. You do not buy their tokens. You buy, or already own, the hardware they run on.

So the comparison that follows is between the 23 models you can actually be invoiced for. Everything about them is the vendor's own published rate, recorded as stated. Where a vendor prices by time of day, we record the peak rate, because that is the number a bill can reach.

The first lever: which side of the request the token is on

Of the 23 priced models, 22 charge more for a token coming out than for the same token going in.

The multiple is remarkably consistent. It runs from 3x to 8x, and the median is exactly 4.0x. Agnes 3.0 Flash, Hy4 Preview, Fugu Max, Qwen3.8-Max, Grok 4.6 and Pareto 26.9 all sit at 3x. DeepSeek V4.1 Flash, o3 and GPT-4o sit at 4x. Claude Sonnet 5, Claude Opus 5, Claude Mythos 5.1, Claude Fable 5.1, GPT-6 Astra, Gemini 3.7 Flash and Gemini 3.8 Flash sit at 5x. Gemini 3.8 Live and its Extended Thinking sibling sit at 6x. Gemini 2.5 Pro is the outlier at 8x.

The single exception is Jev, TypeSafe's typed-answer model, which charges $0.042 per million input tokens and nothing at all for output. It can do that because it does not generate text — it returns a typed answer with a probability attached — so there is barely any output to charge for. That is not a discount. It is a different product.

Now put that next to the size discussion. Shrinking a model from 27 billion parameters to 121 million does not move the 4x. Compressing weights to two bits does not move it. It is not a property of the architecture at all; it is a pricing decision, and it is the most uniform decision on the entire board.

Its practical effect is that the shape of your workload sets your bill more than your choice of model does. Two teams on the identical model, one summarising long documents into short answers and one generating long drafts from short prompts, will see bills that differ by several times over. The first team is buying mostly input. The second is buying mostly output at four times the rate. Nobody in a thread about parameter counts is going to tell you that, and no benchmark measures it.

The arithmetic is worth doing once. Take the median priced model on this board: $1.25 per million in, $6 per million out. A summarisation job that reads 100,000 tokens and writes 1,000 costs about 12.5 cents of input and 0.6 cents of output — roughly 95% input. A drafting job that reads 1,000 tokens and writes 10,000 costs about 0.1 cents of input and 6 cents of output — roughly 98% output. Same model, same rate card: 13.1 cents against 6.1 cents. The second job costs about half as much as the first while moving about a ninth as many tokens, which is to say that per token moved it is roughly four times more expensive. Anyone forecasting spend from a token count alone, without splitting it by direction, will be wrong by a multiple rather than by a margin.

The second lever: the cache, which three vendors out of twenty-three publish

The largest single discount anywhere on this board is not a smaller model. It is the cached-input rate.

Three of the 23 priced models publish one. Agnes 3.0 Flash charges $0.005 per million cached input tokens against $0.05 fresh — ten times cheaper. Pareto 26.9 charges $0.25 against $2.50 — ten times cheaper. Qwen3.8-Omni-Flash charges $0.016 against $0.15 — about nine times cheaper.

Nine to ten times. For comparison, the entire span between the cheapest and the median priced model on this board, on input, is a factor of twenty-five. A cache hit is therefore worth roughly half of that whole journey, and it is available without changing model, vendor or architecture — only by sending the same prefix again.

Any workload with a long fixed prompt — a system prompt, a tool schema, a document held across a conversation, a retrieval context reused across questions — is paying the fresh rate on the same tokens over and over unless the vendor caches and the caller structures the request so the cache can hit.

And for twenty of the twenty-three models on this board, a buyer reading the price field cannot see that rate at all.

The third lever: the tier, which is a 333x decision

Among models you can buy today, the output rate runs from $0.15 per million for Agnes 3.0 Flash to $50 per million for Claude Mythos 5.1, Claude Fable 5.1 and GPT-6 Astra. That is a spread of 333 times across models simultaneously on sale. On input, the spread from $0.05 to $10 is 200 times.

The median priced model on the board charges $6 per million output and $1.25 per million input.

What is worth noticing is how discontinuous that range is. It is not a smooth curve from small and cheap to large and expensive. It is a small number of tiers that vendors place models into, and the placement is a commercial decision that correlates only loosely with anything technical. Gemini 3.7 Flash and Gemini 3.8 Flash carry identical prices a month apart. Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking carry identical prices despite one reasoning while it talks. Claude Mythos 5.1, Claude Fable 5.1 and GPT-6 Astra — three models from two companies — all landed on exactly $10 and $50.

Those are not three independent measurements of what frontier inference costs. That is a price point.

What a small model actually changes about the bill

None of this means the small-model story is wrong. It means it is a different story from the one the price sheets tell, and conflating them is how a buyer gets surprised.

Needle 3, which we added to this board today, has no per-token price and never will. It is Apache-2.0, it is 8 to 29 megabytes, and it runs on a phone, a wearable or a microcontroller. Its inference cost is not lower than $0.15 per million. It is not a per-million number at all. The cost moved: into the hardware that has to exist anyway, into the engineering to integrate it, and into the things it will not do — it does tool calls, structured extraction and embeddings, and by deliberate design it does not chat at all. Cactus's own founder says as much in public.

That is a genuinely good trade for the right workload, and a catastrophic one for the wrong workload, and neither outcome shows up as a price. It is also, worth saying plainly, a model whose headline comparison is disputed in its own launch thread, where one tester reports measuring 32.2% correct tool shapes against another small model's 90.9%.

LimiX-2, which we also added today, makes the point from the other direction. It is a 400-million-parameter model for tables rather than text, its weights are downloadable, and it costs nothing per token. It is also licensed non-commercially. For a business the effective price is not low. It is infinite, until a commercial licence exists.

So the honest summary of the small-model wave, priced rather than benchmarked, is this: it does not make tokens cheaper. It replaces a token price with a set of costs that no price table on this board, or anyone else's, can show you. That may well be the better deal. It is not the same claim, and the threads are making the first claim while citing evidence for the second.

What this board cannot see, including where it is our own fault

Yesterday we published a piece arguing that as models get smaller and more specialised, the evidence about them stops being comparable. This is the same board's price column, and it has the same problem, including a share that belongs to us.

The "three of twenty-three" figure measures what we have stored, not what vendors disclose. Anthropic publishes cache pricing — our own launch scorer carries a note that Claude Fable 5.1 held headline pricing flat while cutting cache reads by 75%, which is precisely a cache-price disclosure. That is not in the structured price field for those models, so it is not in the count. The correct reading of "three publish a cached rate" is "three have a cached rate recorded here". The direction of the finding survives; the precision does not.

Fifty-seven rows carry no structured specification at all. They are older entries — GPT-4o mini, o1, GPT-4 Turbo, Claude 3 Opus, Gemini 2.0 Flash, Llama 4 Maverick and others — added before the field existed. They are not priced here, and this article does not claim anything about them.

Twenty-seven models were released in September 2026, and 14 of them publish a price. Slightly better than half. That is the most current statement this board can make about pricing disclosure, and it is not a flattering one for the industry.

Every figure above is a vendor's own published rate. Nobody here has been invoiced by these vendors and checked the bill against the rate card. A published price is a claim, like a benchmark is a claim.

Where our own products sit in these numbers

TrustList's own Rutba products are listed on the launch board, and we disclose that whenever the board is used as evidence. None of them is an AI model, so none of them appears in the model board counts in this article. Recomputing every figure above with our own listings removed changes nothing, because none was ever in them.

What to ask a vendor

If there is a practical answer to what makes a token expensive, it is this. The size of the model is the last thing to ask about, not the first.

Ask for the output rate, not the headline rate. The headline number in most announcements is the input rate, and you will spend a median of four times that on everything the model says back. Estimate your own input-to-output ratio before comparing any two vendors, because that ratio, not the model, is what your bill is most sensitive to.

Ask for the cached-input rate, and whether your request shape can hit it. It is worth nine or ten times on the models that publish one, and it is the largest discount available anywhere on this board. If a vendor does not publish one, that is itself an answer.

Ask which tier the model is in and what else is in that tier. A 333x spread across models on sale at the same moment is not a technical gradient. Two models at the same price are making the same commercial claim about themselves, and one of them is usually wrong.

And if the answer is an open-weights model, stop asking about tokens. Ask about the hardware, the integration, the licence, and what the model refuses to do. Needle 3 will not chat. LimiX-2 will not be used commercially. Those are the prices. They are simply not quoted per million.