# AI Infrastructure: The Hidden Economics of Computing Power

Author: Norbert Biedrzycki
Published: 2022-12-15
Language: en
URL: https://norbertbiedrzycki.pl/en/ai-infrastructure-the-hidden-economics-of-computing-power/
Summary: Software investors were trained to love 80 percent gross margins. AI is bringing back an old variable they had almost forgotten: the cost of the machine.

**Software investors were trained to love 80 percent gross margins. AI is bringing back an old variable they had almost forgotten: the cost of the machine.**

For two decades, the economics of software were wonderfully simple. You built a product once, sold it many times, and the marginal cost of an additional customer was close to zero. Cloud computing made infrastructure an operating expense rather than a capital investment, and most software investors stopped thinking about hardware altogether.

Large AI models are changing this.

**Compute is becoming the cost of goods sold**

Every time someone asks ChatGPT a question, a cluster of GPUs has to process the request. Every new model generation requires a training run that consumes enormous computing power for weeks. Sam Altman recently described the compute costs of running ChatGPT as "eye-watering". For companies building on these models, the cost of inference is starting to look less like hosting and more like cost of goods sold.

This changes the questions investors must ask. Not only what is the gross margin today, but how does it scale with usage? What is the cost per query, and how fast is it falling? How dependent is the company on a single cloud provider for capacity? What happens if GPUs become scarce?

**GPUs as strategic assets**

This year has also shown that computing capacity is becoming geopolitical. In October, the US government introduced sweeping export controls restricting the sale of advanced AI chips and chipmaking equipment to China. Nvidia's most advanced data centre GPUs, the A100 and the newly announced H100, fall under these rules. Semiconductor supply chains are now an explicit instrument of national strategy.

At the same time, research published by DeepMind this spring, the so called Chinchilla paper, showed that many large models had been trained inefficiently and that more data, not just more parameters, could deliver better performance for the same compute. This is an important reminder that efficiency gains can shift the economics quickly, and that today's assumptions about computing requirements may not hold for long.

**Where the opportunities sit**

From an investment perspective the opportunity extends well beyond chip designers.

Cloud providers will benefit from AI workloads, but they will also have to invest heavily in specialised hardware. Data centre operators may find that AI changes the design of facilities: higher power density, liquid cooling, different locations. Companies that improve utilisation of expensive hardware, through better scheduling, model compression or more efficient inference, could become valuable precisely because compute is expensive. And there is a growing market for specialised chips designed for particular AI workloads.

As someone who has managed large IT services organisations, I know how often infrastructure capacity is either underused or unavailable at the moment it is needed. In AI, both errors are costly. Underused GPUs destroy returns. Unavailable GPUs destroy time to market.

**The strategic implication**

I believe we are entering a period in which control over computing capacity will become a competitive advantage in its own right. Not for every company, since most businesses will rent rather than own. But for those competing at the frontier, access to compute could be as important as access to talent or data.

For boards this has a practical implication. AI strategy can no longer be discussed only in terms of use cases. It must also be discussed in terms of cost per transaction, infrastructure dependency and supplier risk.

*Investment question: Could control over computing capacity become one of the most important competitive advantages in the AI economy?*
