AI chips in 2024: what Sam Altman wants to spend trillions on

Artificial intelligence is already becoming integral to most spheres of activity. Alongside advances in algorithms, hardware—especially specialised chips—plays a decisive role.

Modern AI methods depend on computations at a scale that was impossible only a few years ago. Training cutting-edge algorithms can take months and cost tens of millions of dollars.

Such colossal compute is delivered by specialised chips packed with as many transistors as possible and designed to perform the calculations AI systems require efficiently.

This article examines the history, operating principles and spread of AI chips, their advantages in performance and energy efficiency over prior generations and general-purpose chips, and the semiconductor-industry and chip-design trends shaping the sector’s evolution.

By “AI chips” we mean specialised computer chips that achieve high efficiency and speed for specific computations at the expense of performance on other kinds of workloads.

• Specialised AI chips deliver higher performance and energy efficiency than general-purpose processors.
• Leading firms such as NVIDIA, Intel, AMD, Microsoft, Amazon and Google are investing heavily in specialised AI chips.
• Adoption of advanced process nodes will enable more compact, faster and more energy‑efficient AI chips.
• AI chips are beginning to appear in consumer electronics such as smartphones and PCs to run AI tasks directly on users’ devices.

Industry favours AI chips 

From the 1960s to the 2010s, engineering innovations that shrank transistors allowed their number on a single chip to double roughly every two years—a phenomenon known as Moore’s law. As a result, processor performance and energy efficiency rose by orders of magnitude.

Today, however, transistor features are only a few atoms across. Shrinking them further creates formidable engineering challenges, driving up capital expenditure and the cost of highly skilled labour in the chip industry.

As a result, Moore’s law is slowing, lengthening the time required to double transistor density. Persisting with scaling remains worthwhile, though: it keeps improving efficiency and speed, and allows more specialised circuits to be integrated on a single die.

Historically, scaling gains favoured general-purpose chips such as central processing units. Rising demand for specialised solutions—particularly for AI—has broken that pattern. Specialised AI chips are now taking share from universal processors.

The basics of AI chips

AI chips include:

• graphics processing units (GPUs);
• field-programmable gate arrays (FPGA);
• application-specific integrated circuits (ASICs) for AI.

General-purpose central processing units (CPUs) can also be used for some simple AI tasks. As technologies advance, however, their use becomes less efficient.

Like general-purpose processors, AI chips gain speed and energy efficiency by packing ever more, ever smaller transistors. Unlike CPUs, they add architectural features optimised specifically for AI workloads.

These features accelerate the repetitive, predictable and independent computations AI algorithms need. They include:

• performing large numbers of operations in parallel rather than sequentially;
• reduced-precision arithmetic to implement AI algorithms successfully while cutting the required transistor count;
• faster memory access, for example by storing an entire AI model on one chip;
• programming languages designed to translate AI code efficiently for execution on the target chip.

Different AI chips suit different tasks. GPUs are most often used for initial development and training, FPGAs for running trained models on real data (inference), and ASICs can be designed for both training and inference.

Market potential 

AMD chief executive Lisa Su put the total addressable market for AI chips at $400bn. To put the figure in context, New Street Research analysed the industry’s current state.

For now, NVIDIA dominates, with $38-39bn of spending, followed by Broadcom, which designs TPUs for Google. Despite rapid growth in AI accelerators, GPUs are expected to account for the bulk of market expansion by 2027.

New Street Research forecasts substantial growth in AI chips over the next few years. By 2027 the market could reach a scale equivalent to roughly 10m servers housing 100m chips. That corresponds to about 10% of internet users employing AI for various purposes.

The embedding of AI into everyday applications and workflows will encourage its use and propel the AI‑chip industry.

Industry leaders recognise the sector’s potential. According to media reports, OpenAI head Sam Altman has led an effort to raise up to $7trn for semiconductor expansion to ensure future AI development has the necessary compute.

The initiative may not succeed. But it shows how seriously technology visionaries take the AI race.

The impact of COVID-19 and generative AI 

At the start of the COVID-19 pandemic the semiconductor industry sank into a severe crisis as supply chains faltered. Every sector dependent on chips—from carmaking to entertainment and consumer electronics—was hit.

By 2022 the situation began to stabilise. Leading manufacturers adapted to the new reality, and some pandemic restrictions were lifted. The launch of ChatGPT in late November 2022, however, heaped fresh pressure on a just-recovering industry.

The staggering success of a service from a small start-up did not go unnoticed by tech giants. Microsoft, Google, Amazon, Meta and many others entered a new race to develop generative AI.

The chip industry now risks another painful shortage. Demand for AI chips is extraordinary; producers are selling future batches months ahead, and a plethora of products is arriving from both established and lesser-known firms.

The main players 

Today NVIDIA leads in chips for AI training, with an 80% share. Since early 2023, the boom in generative AI has sent the firm’s market capitalisation from $364bn to $2.19trn.

Firms such as Intel and AMD are also investing billions of dollars in specialised AI chips as they seek a place in this promising market.

Cloud providers including Microsoft, Amazon and Google are building their own AI chips. Though still NVIDIA customers, the tech giants want to reduce reliance on external suppliers and secure supply amid looming shortages.

Other players are in the market too, mostly with narrower offerings. They include Kneron, Cerebras Systems and Lightmatter.

Crucially, few foundries can manufacture at cutting-edge nodes. The companies above typically order semiconductors from TSMC, Intel and Samsung.

What next

The specialised AI‑chip industry is set for rapid development. Key drivers will be rising demand, adoption of advanced process nodes and large-scale investment in semiconductors. Leading players will focus on innovation.

In 2024–2025 TSMC will start shipping initial batches and small‑run production of 2nm chips. Mass production is expected in the second half of 2025.

The move to 2nm targets energy efficiency. Transistor speed is expected to rise by 10–15% at the same power, or power consumption to fall by 20–30% without sacrificing performance.

In 2026 TSMC will move to a second‑generation 2nm N2P process with backside power delivery. After mastering 3nm and 1.4nm nodes, Intel plans to develop a 1nm process, where 1nm is just four silicon atoms across.

AI chips in consumer electronics are also set to advance. Manufacturers such as Intel, AMD, Apple and Qualcomm are adding specialised neural processing units (NPUs) to their products. These will run AI tasks directly on users’ devices.

For example, Qualcomm’s current Snapdragon 8 Gen 3 already can deploy models such as Stable Diffusion and Meta Llama 2. Developers say it can generate an image in under a second—the previous technology took roughly 15 seconds. By comparison, a well‑equipped laptop may need up to two minutes to create a picture.

In sum, continual improvements in manufacturing will yield ever smaller, faster and more energy‑efficient AI chips. Heavy investment in this promising sector will spur a wave of innovation.

Text: Bohdan Kaminsky