The Biggest Lie About AI

“I’m not sure that AI can solve the problems of inequality and economic growth.”

And, strangely enough, he not only demonstrates remarkable foresight but may inadvertently exacerbate the problem, as the company has set a new record for price increases with its new “Fast Mode,” which costs a staggering $150 per million output tokens — literally more than ten times more expensive than most alternatives and completely out of reach for most people.

Today we debunk the myth that advanced AI is neither cheap nor becoming cheaper, explaining why this is actually not the “fault” of AI laboratories, the true and alarming reason why prices are not falling, and the consequences that this, as I humbly believe, has for society.

Understanding AI Cost Trends

Since the release of ChatGPT in November 2022, AI has experienced a couple of years of intense price deflation, to the point that some industry representatives, such as Google CEO Sundar Pichai, predicted that we would get “intelligence as air, too cheap to matter.”

At that time (the end of 2024), token prices had fallen by 97% over the last 18 months, creating the impression that prices were plummeting... to zero.

But was that really the case? No.

And to explain this, we must return to the most basic unit of AI: the token.

What is a token?

In generative AI, which has absorbed not only a large portion of investments but also a significant part of AI usage these days, everything is measured in “tokens.”

A token is simply the most basic semantic unit of information in a data modality. In text, it is words or syllables; for image models, it is fragments of pixels, for video — fragments of pixels across several frames, etc.

In practice, everything you send to a generative AI model is "tokenized," broken down into these units of information that the AI can then process. As you can see below, the sequence "This was such an interesting article!" is broken down into seven tokens for processing by the DeepSeek R1 model, shown in different colors.

Naturally, the response is also a combination of tokens that collectively represent the entire interaction.

You are then charged based on the number of input tokens you provided to the model (in this case, seven), as well as the number of tokens it generates in response to you.

This seems like a fair way to charge, but the truth is that it's a very opaque system where not only do you not easily see or predict how many tokens you're spending, as we will see later, but each model and company differs significantly.

But before we open this Pandora's box, we must consider another key factor: computations during inference.

Cost is the result of two factors

It's one thing how much you pay for a processed or generated token; it's another how many tokens were in that specific interaction.

That is, the actual US dollars or Korean won you pay are the result of multiplying these unit values by the total quantity:

(number of input tokens × price per million input tokens) + (number of output tokens × price per million output tokens) = how much you pay.

This means you need to consider not only the unit price your provider offers but also two other factors: "model verbosity" and "token compression."

• Model verbosity represents how "talkative" the models are. For example, Claude from Anthropic generates a lot, and I mean a lot, of tokens.

• Token compression refers to the average size of a token. If the model's tokenizer breaks a sequence into 8 tokens instead of 4, you are literally paying twice as much for the same amount of text (assuming we are talking about chatbots with equal unit prices).

Returning to our previous example, based on the token price, Google's Gemma model required one more token for the same sequence than DeepSeek R1. This means that assuming the unit prices of tokens were the same, Gemma would be a more expensive model to process and likely a more expensive model for inference because it would generate more tokens on average per response.

Of course, Gemma is a much smaller model and therefore cheaper to maintain, so AI labs will quote you a lower unit price.

Consequently, the easiest thing for most of us is simply to purchase subscriptions that charge you a fixed monthly price regardless of usage.

This, of course, is also not a fair game, because if you don't use the model at all, you are still charged $20/month or even $200/month for higher tiers. But if you use the models too much, they impose a "rate limit," literally preventing you from using the models more, or they will subtly downgrade you to worse models.

And why is all this important? Because, if you remember, we said that chatty models significantly affect your costs. And well, describing modern state-of-the-art models as "chatty" is a huge understatement, as these models can generate millions of tokens in a single session, and anywhere from 20 to 100 times more tokens than the models that existed when ChatGPT first entered our lives.

This is because AI models benefit from longer generations of tokens. That is, since they need to "speak to think," the more they "speak," the more they "think." And the more they "think," the better the results.

This is undoubtedly the main driver of progress in AI today, which means our models are generating more tokens every day.

In short, even if unit prices for tokens continue to fall, this decline is always easily offset by a sharp increase in the number of tokens generated.

There are definitely some studies being conducted to increase the "intelligence per token" of models (that is, making models "as good" without being so verbose), but unfortunately, they are overshadowed by the enormous number of tokens that these models produce every time you interact with them.

But all of this shouldn't matter, because prices are dropping so much that you really don't care, right?

Fast Mode... or rather, Unavailable Mode

Listen, I am an advanced user when it comes to AI. I am literally the embodiment of inelastic demand. I am the example you will see in economics textbooks, a person so willing to pay that you can treat me poorly, and I will still end up paying.

And even I couldn't help but look on in horror when I saw the prices for Anthropic's new "Fast Mode."

With the promise of 2.5 times faster responses, Fast Mode costs six times more than the previous most expensive option, Claude, which is already the most expensive model available today. In comparison, this new model is more than ten times more expensive than GPT-5.2.

But the truth is that while Anthropic is an exception, this has been the reality of AI pricing for quite some time.

The Myth of Price Deflation

During the period from 2022 to 2026, the list prices for key usage-based LLM APIs showed a pricing model that has nothing to do with the widespread belief that prices are falling:

We had initial large stepwise reductions in the cost of "state-of-the-art" capabilities until mid-2024, but they were followed by a shift to price segmentation (tiers, caching, batch processing) and ultimately selective price increases for higher reasoning/"pro" options.

An example of this is OpenAI's shift from the initial launch price of the GPT-4 API at $60 per million output tokens in March 2023 to $10 per million output tokens for later versions of GPT-4o, then to the release of GPT-5 at $10 per million tokens (August 2025), and then back up to $14 per million output tokens for GPT-5.2 (December 2025).

The path of Anthropic is even more aggressive: in May 2023, the published price list indicated Claude-v1 at $11.02 per million tokens for requests and $32.68 per million tokens for completions; Claude 3 Opus was launched in March 2024 at $15 per million input tokens and $75 per million output tokens; and the current price list shows Claude Opus 4.6 at $5 per million input tokens and $25 per million output tokens, but now including the already infamous fast mode.

The only major American lab that showed more stable prices is xAI of Elon Musk, as well as Chinese labs, which, typical of Chinese market dynamics where hyper-competition is promoted, have fierce competition that has led to very cheap token prices.

However, their lack of computational power does not allow them to compete at the product level (for instance, comparing the DeepSeek app with ChatGPT). Contrary to popular belief, the "U.S. advantage" is definitely not at the model level, where models are of roughly equal quality, but at the computational level during inference (allowing them to think longer).

But the real question here, aside from Anthropic's need to make more money (they are certainly free to set the prices they want), is whether they are "forced" to set such prices.

And the answer to that, I believe, is yes, due to how they implemented it.

The Magic of Batch Processing

Aside from the price, the most confusing thing about all this is that Anthropic suddenly released a model that is just as good as slower models, but much faster.

How?

Usually, "better" means slower, because models are either larger or think longer, so "better and faster" doesn’t make much sense. Thus, the most likely explanation is the magic of batch processing.

That is, they send your requests in smaller batches to the GPU/TPU. Let me explain.

When you hit "Send" in ChatGPT or Claude, your request is sent to an XPU server (a group of accelerators like GPU, TPU, or LPU) for processing. Typically, your request is processed alongside requests from other users.

But why? Well, AI models require an absurd amount of computation for a single prediction.

I am greatly simplifying here, but if you have a dense model with a trillion parameters (modern state-of-the-art models are much larger these days, although sparse), this model requires on the order of 2×N computations, where N is the number of activated parameters. Therefore, if we assume a dense model, this model requires 2 trillion operations for a single prediction.

During inference, we cache a lot of computations during the decoding phase (the phase when you see a stream of new words appearing on your screen), which significantly reduces computational requirements. But the essence remains that we still need a lot of computational power for each prediction.

Fortunately for us, top GPUs these days are actually extremely overloaded with computations for AI (the biggest reason we need so many GPUs is actually memory constraints, not computational constraints), so they can not only handle this prediction in a fraction of a second, they can actually make predictions for multiple sequences in parallel, really quickly.

In practice, this means that for the cost of one pass of the model forward (the computations required for prediction), you get dozens of them, one for each sequence in the batch.

This means that NVIDIA GPUs are designed for parallelization, so the larger each batch sent to the system, the higher the hardware utilization.

When you consider that these accelerators are extremely expensive to purchase and maintain, the importance of high hardware utilization becomes even more critical for making money on these investments.

However, it also implies slower performance because the matrices (or, more technically speaking, 3D tensors, since we also have a batch dimension) are larger as a result.

In simple terms, you can choose to serve 50 people at once instead of serving one person at a time, but each person will get their answer more slowly. And, as you can probably guess, in the world of digital products, latency is the biggest killer of user experience.

But wait, you have another option. You can simply... serve one client, but make them pay much more to account for the extremely small sizes of the packages. From my limited understanding (I don't work at Anthropic), I believe this is exactly what is happening here:

Anthropic offers you the "opportunity" to run a whole server primarily to serve you, with the advantage of very fast responses and a very interesting discussion with your boss next month about your "AI bills."

And what does all this mean? Well, more inequality that is not going to improve anytime soon, because the main driver of costs is not under the control of these labs, I fear, and it reveals a lot about what is going on behind the scenes.

Check the capabilities of different AI models

Speaking of prices and performance of AI models — there is a way to evaluate various models in practice without overpaying. Understanding which model really fits your tasks can save significant funds.

I will say it straight: as long as energy constraints and markups on equipment remain, AI will stay expensive and will become even more expensive.

This is simply a transfer of costs, or "who will foot the bill." NVIDIA charges a 4x markup compared to production costs to this day, which means the price tag is 4 times the cost of manufacturing GPUs — we know this from NVIDIA's 75% gross margin.

This puts companies like Anthropic, OpenAI, or hyperscalers in an impossible position to make money; they overpay for a product that is almost always serviced suboptimally (too long to explain here, but AI inference is known to be inefficient in servicing), while constantly being under pressure from competition in what is essentially a commodity industry, except for who owns the large computational power and data.

That is, it is not that the AI models themselves are differentiated; it’s the extent to which you, as a lab, have the necessary computational power to run these models with long inference budgets.

Incredibly, these labs have managed to launch these models with quite decent gross profits, but their capital expenditures and research and development (R&D) costs kill any hope of making a profit.

We know this mainly from the prospectuses of Chinese labs Minimax and Zhipu Labs, which recently conducted IPOs. The dynamics are identical; it’s just “the same but bigger” in the U.S.

In simple terms, the costs of staying ahead do not allow them to make money, even if they are ahead. The problem?

This can only get worse as demand for AI services grows faster than production capacities can keep up due to energy constraints.

In simple terms, soon the demand for AI will outstrip providers’ ability to bring new computational capacity online due to energy constraints (i.e., lack of places to plug in number-crunching machines).

This leads to my prediction: top AIs, the best possible models, will become more expensive over time, risking creating a real divide between those who can and cannot afford them.