Ways to think about token pricing

https://www.ben-evans.com/benedictevans/2026/7/9/ways-to-think-about-token-pricing

by mercutio2 • 13 days ago

46 23 13 days ago

ben-evans.com

nekusar2• 13 days ago

1. Slot machine

You put in minutes of time combined with token costs. You pull the lever and HOPE something good comes back.

And it gets tantalizingly close, but for 100% gotta pull the arm again!

shshsjsj• 13 days ago

that is way too accurate and ofc hn would downvote something funny and true.

techgnosis• 13 days ago

it just seems off-topic. we see the slot machine analogy a lot but i dont see how it has anything to do with this long, thoughtful essay about pricing power

mips_avatar4• 13 days ago

I know there's a lot of reasons to think that everyone will just use AI inference in the cloud, but I think if everyone had access to a dgx gb400 class machine with 512gb of hbm4 vram and 1tb of lpddr8x a lot of people are going to be running finetunes of models locally. Like the dgx gb300 is $94k now, but i bet this class of machine will come down to $20kish in the next 2-3 years.

cyanydeez2• 13 days ago

sure, if there werent capitalists digging their moat. in 1 year local inferwnce doubled cost from ram alone because cloud vendors cornered markets on their circular cash flow expectations.

mips_avatar4• 13 days ago

Yeah but Samsung/Micron/SK hynix are building new fabs, and Chinese Ram will be able to fully supply the lower end ddr5 ram market within 2 years. The crunch is temporary and on the other side of it there will be incredible hardware at the $10-20k price point.

NuclearPM2• 13 days ago

There’s a shitload of money to be made in RAM fabrication. Costs will come down a ton.

Analemma_• 13 days ago

You are waaaaaay too optimistic about RAM prices. This is not a one-quarter crisis and you need to settle in for the long haul: the era of "hardware always comes down in price" is over, definitely for the next 3 years and possibly the next 5-8. The new DRAM fabs aren't even coming online until 2028 and there's not much reason to think they'll make much of a dent in the supply-demand mismatch even when they do.

Legend2440• 13 days ago

DRAM prices have been highly cyclical for a long time. They'll come down again.

mips_avatar4• 13 days ago

The marginal cost of making RAM is zero and it's clear to everyone that there's near infinite demand, so the fab capacity is coming. Maybe the high end won't expand fast enough, but you can do a lot with Chinese ddr5.

anthonypasq• 13 days ago

if all the competition pushes down margins on tokens to 10-20%, i dont see how the inherent scale advantages of cloud inference wont be way more to account for the 20% cheaper tokens youd be getting running locally. i dont see how local will ever be more economical

pixl97• 12 days ago

What is the economic cost of having your data monitored by external sources?

typ• 13 days ago

Tokens will surely become commodities, but models don't need to.

The point is that electricity becomes a commodity, rather than, say, nuclear reactors or gas turbines also need to be commoditized.

Contrary to most people in the software-minded circle, I am not very excited about running local LLMs. Most of what I have seen involves selling wrappers that call APIs on top of an LLM AI, similar to how traditional SaaS has generated revenue from new technologies. So, this would certainly make those people who are working on that sensitive about the pricing. LLM pricing is treated as a continuous cost in the COGS. They don't really use AI to create much value (consumer surplus) other than replacing the existing boring products, like just building the good old CRUD apps, but faster.

What I find REAL interesting about the potential of LLM AI instead is to create new technology out of it, or revolutionize something old to be order-of-magnitude more efficient. In this regard, the expenses on those tokens are more akin to an upfront CapEx.

Cheaper tokens would surely be nice. But if what we are talking about is, like, solving self-driving, curing cancer, or making air-conditioners 100x efficient, the narrow focus on running a cheaper model in my home so I can write my SaaS apps for my get-rich-quick business looks really unpromising and a waste of time for human civilization in a near-singularity horizon.

t0mpr1c32• 12 days ago

Agreed, although most of what humans do is pretty pointless in that context. Even if AI turns out to be a multiplier that makes everyone quicker and smarter, people will mostly just want to use it to make more money than the next guy. The history of capitalism suggests that the money-go-round will create useful technology along the way.

t0mpr1c32• 12 days ago

I agree with the author's overall conclusion about pricing LLMs, although I question some of the reasoning. He does not talk enough about one important input: the data used to train the models. To a first approximation, their utility is a function of the quantity of data available. Currently, the bulk of the training data is public (the internet) and that is a major reason why the performance of different models is not diverging more. One huge future area for AI is modeling the physical world and in that arena people are going to be creating their own empirical datasets. Waymo has achieved FSD and Tesla has not. It is not coincidental that Waymo has built a sophisticated platform to simulate the physical world.

moriwo-dev-ai• 11 days ago

A practitioner data point for the first question (how many use cases need the frontier): I run a coding agent split across model tiers, and measured that roughly 80% of calls (file reads, routine edits, summarization) were handled fine by mid-tier models, with the frontier model only doing planning and review — cost roughly halved with no quality drop I could detect. The interesting failure was calibration: when I had the frontier model grade the cheaper models' work against its own standards, almost everything escalated back to the frontier and the savings vanished. So the demand for frontier tokens isn't a fixed property of the use case — it depends heavily on how the pipeline around it is designed. My guess is "good enough" eats even more of that curve than it looks like today.

gawkdev• 11 days ago

price per token on its own tells you nothing.. what matters is tokens per successful outcome a cheap model that needs three retries costs more than an expensive one that gets it right first go.

anyone actually tracking cost per completed task rather than cost per call ? it is quite a difficult task to define completion as it is a qualitative aspect and can vary from every individual.

Ways to think about token pricing

Related Stories