{"id":11810,"date":"2024-03-20T17:15:00","date_gmt":"2024-03-20T15:15:00","guid":{"rendered":"https:\/\/forklog.com\/en\/ai-chips-in-2024-what-sam-altman-wants-to-spend-trillions-on\/"},"modified":"2024-03-20T17:15:00","modified_gmt":"2024-03-20T15:15:00","slug":"ai-chips-in-2024-what-sam-altman-wants-to-spend-trillions-on","status":"publish","type":"post","link":"https:\/\/forklog.com\/en\/ai-chips-in-2024-what-sam-altman-wants-to-spend-trillions-on\/","title":{"rendered":"AI chips in 2024: what Sam Altman wants to spend trillions on"},"content":{"rendered":"<p>Artificial intelligence is already becoming integral to most spheres of activity. Alongside advances in algorithms, hardware\u2014especially specialised chips\u2014plays a decisive role.<\/p>\n<p>Modern AI methods depend on computations at a scale that was impossible only a few years ago. Training cutting-edge algorithms can take months and cost tens of millions of dollars.<\/p>\n<p>Such colossal compute is delivered by specialised chips packed with as many <span data-descr=\"Basic computing devices that can switch between 'on' (1) and 'off' (0) states\" class=\"old_tooltip\">transistors<\/span> as possible and designed to perform the calculations AI systems require efficiently.<\/p>\n<p>This article examines the history, operating principles and spread of AI chips, their advantages in performance and <span data-descr=\"The ability to perform more computations per unit of energy consumed\" class=\"old_tooltip\">energy efficiency<\/span> over prior generations and general-purpose chips, and the semiconductor-industry and chip-design trends shaping the sector\u2019s evolution.<\/p>\n<p>By \u201cAI chips\u201d we mean specialised computer chips that achieve high efficiency and speed for specific computations at the expense of performance on other kinds of workloads.<\/p>\n<ul style=\"background-color:#ebffff\" class=\"has-background wp-block-list\">\n<li>Specialised AI chips deliver higher performance and energy efficiency than general-purpose processors.<\/li>\n<li>Leading firms such as NVIDIA, Intel, AMD, Microsoft, Amazon and Google are investing heavily in specialised AI chips.<\/li>\n<li>Adoption of advanced process nodes will enable more compact, faster and more energy\u2011efficient AI chips.<\/li>\n<li>AI chips are beginning to appear in consumer electronics such as smartphones and PCs to run AI tasks directly on users\u2019 devices. <\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\"><strong>Industry favours AI chips\u00a0<\/strong><\/h2>\n<p>From the 1960s to the 2010s, engineering innovations that shrank transistors allowed their number on a single chip to double roughly every two years\u2014a phenomenon known as <a href=\"https:\/\/ru.wikipedia.org\/wiki\/%D0%97%D0%B0%D0%BA%D0%BE%D0%BD_%D0%9C%D1%83%D1%80%D0%B0\">Moore\u2019s law<\/a>. As a result, processor performance and energy efficiency rose by orders of magnitude.<\/p>\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh7-eu.googleusercontent.com\/_8BF8PND4zUqag8b8kXRyiAigQz7uSMwg1nP3-FbnDL3wzHfyBi3XARpjxvrB2vb0ktA4icw5vOagGo8xywEjj8Lsw_rN3hGf_1JCW5w6lavmARCzY6XJmVEDWnY9jJf-_-bj3tCI1fzTrtY2aEAOmQ\" alt=\"\u0418\u0418-\u0447\u0438\u043f\u044b \u0432 2024 \u0433\u043e\u0434\u0443: \u043d\u0430 \u0447\u0442\u043e \u0421\u044d\u043c \u0410\u043b\u044c\u0442\u043c\u0430\u043d \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u0442 \u0442\u0440\u0430\u0442\u0438\u0442\u044c \u0442\u0440\u0438\u043b\u043b\u0438\u043e\u043d\u044b\"\/><figcaption class=\"wp-element-caption\">Processor performance improvements, normalised to 1979. Source: <a href=\"https:\/\/ourworldindata.org\/technological-change\">Our World in Data<\/a>.<\/figcaption><\/figure>\n<p>Today, however, transistor features are only a few atoms across. Shrinking them further creates formidable engineering challenges, driving up capital expenditure and the cost of highly skilled labour in the chip industry.<\/p>\n<p>As a result, Moore\u2019s law is slowing, lengthening the time required to double transistor density. Persisting with scaling remains worthwhile, though: it keeps improving efficiency and speed, and allows more specialised circuits to be integrated on a single die.<\/p>\n<p>Historically, scaling gains favoured general-purpose chips such as central processing units. Rising demand for specialised solutions\u2014particularly for AI\u2014has broken that pattern. Specialised AI chips are now taking share from universal processors.<\/p>\n<h2 class=\"wp-block-heading\"><strong>The basics of AI chips<\/strong><\/h2>\n<p>AI chips include:<\/p>\n<ul class=\"wp-block-list\">\n<li>graphics processing units (GPUs);<\/li>\n<li>field-programmable gate arrays (<a href=\"https:\/\/ru.wikipedia.org\/wiki\/%D0%9F%D1%80%D0%BE%D0%B3%D1%80%D0%B0%D0%BC%D0%BC%D0%B8%D1%80%D1%83%D0%B5%D0%BC%D0%B0%D1%8F_%D0%BF%D0%BE%D0%BB%D1%8C%D0%B7%D0%BE%D0%B2%D0%B0%D1%82%D0%B5%D0%BB%D0%B5%D0%BC_%D0%B2%D0%B5%D0%BD%D1%82%D0%B8%D0%BB%D1%8C%D0%BD%D0%B0%D1%8F_%D0%BC%D0%B0%D1%82%D1%80%D0%B8%D1%86%D0%B0\">FPGA<\/a>);<\/li>\n<li>application-specific integrated circuits (ASICs) for AI.<\/li>\n<\/ul>\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" data-id=\"228815\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/image-653.webp\" alt=\"image-653\" class=\"wp-image-228815\"\/><figcaption class=\"wp-element-caption\">An NVIDIA graphics processor. Source: <a href=\"https:\/\/www.nvidia.com\/ru-ru\/data-center\/a100\/\">NVIDIA<\/a>.<\/figcaption><\/figure>\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"638\" data-id=\"228813\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/1503444863118-1024x638.jpg\" alt=\"1503444863118\" class=\"wp-image-228813\" srcset=\"https:\/\/forklog.com\/wp-content\/uploads\/1503444863118-1024x638.jpg 1024w, https:\/\/forklog.com\/wp-content\/uploads\/1503444863118-300x187.jpg 300w, https:\/\/forklog.com\/wp-content\/uploads\/1503444863118-768x479.jpg 768w, https:\/\/forklog.com\/wp-content\/uploads\/1503444863118.jpg 1280w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption class=\"wp-element-caption\">Xilinx field-programmable gate arrays (FPGA). Source: <a href=\"https:\/\/www.xilinx.com\/products\/boards-and-kits\/ek-s6-sp605-g.html\">AMD Xilinx<\/a>.<\/figcaption><\/figure>\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" data-id=\"228814\" src=\"https:\/\/forklog.com\/wp-content\/uploads\/tpu-v2-3.2e16d0ba.fill-1592x896-1200x675-1.webp\" alt=\"tpu-v2-3.2e16d0ba.fill-1592x896-1200x675\" class=\"wp-image-228814\"\/><figcaption class=\"wp-element-caption\">A Google Cloud TPU board with four ASICs, delivering 180 teraflops. Source: <a href=\"https:\/\/blog.google\/products\/google-cloud\/google-cloud-offer-tpus-machine-learning\/\">Google<\/a>.<\/figcaption><\/figure>\n<\/figure>\n<p>General-purpose central processing units (CPUs) can also be used for some simple AI tasks. As technologies advance, however, their use becomes less efficient.<\/p>\n<p>Like general-purpose processors, AI chips gain speed and energy efficiency by packing ever more, ever smaller transistors. Unlike CPUs, they add architectural features optimised specifically for AI workloads.<\/p>\n<p>These features accelerate the repetitive, predictable and independent computations AI algorithms need. They include:<\/p>\n<ul class=\"wp-block-list\">\n<li>performing large numbers of operations in parallel rather than sequentially;<\/li>\n<li>reduced-precision arithmetic to implement AI algorithms successfully while cutting the required transistor count;<\/li>\n<li>faster memory access, for example by storing an entire AI model on one chip;<\/li>\n<li>programming languages designed to translate AI code efficiently for execution on the target chip.<\/li>\n<\/ul>\n<p>Different AI chips suit different tasks. GPUs are most often used for initial development and training, FPGAs for running trained models on real data (inference), and ASICs can be designed for both training and inference.<\/p>\n<figure class=\"wp-block-table\">\n<table>\n<tbody>\n<tr>\n<td rowspan=\"2\">\u00a0<\/td>\n<td colspan=\"2\">Training<\/td>\n<td colspan=\"2\">Inference<\/td>\n<td rowspan=\"2\">Generality<\/td>\n<td rowspan=\"2\">Inference accuracy<\/td>\n<\/tr>\n<tr>\n<td>Efficiency<\/td>\n<td>Speed<\/td>\n<td>Efficiency<\/td>\n<td>Speed<\/td>\n<\/tr>\n<tr>\n<td>CPU<\/td>\n<td colspan=\"4\">1\u00d7 baseline<\/td>\n<td>Very high<\/td>\n<td>~98-99.7%<\/td>\n<\/tr>\n<tr>\n<td>GPU<\/td>\n<td>~10-100x<\/td>\n<td>~10-1000x<\/td>\n<td>~1-10x<\/td>\n<td>~1-100x<\/td>\n<td>High<\/td>\n<td>~98-99.7%<\/td>\n<\/tr>\n<tr>\n<td>FPGA<\/td>\n<td>\u2014<\/td>\n<td>\u2014<\/td>\n<td>~10-100x<\/td>\n<td>~10-100x<\/td>\n<td>Medium<\/td>\n<td>~95-99%<\/td>\n<\/tr>\n<tr>\n<td>ASIC<\/td>\n<td>~10-1000x<\/td>\n<td>~10-1000x<\/td>\n<td>~10-1000x<\/td>\n<td>~10-1000x<\/td>\n<td>Low<\/td>\n<td>~90-98%<\/td>\n<\/tr>\n<\/tbody>\n<\/table><figcaption class=\"wp-element-caption\">A comparison of contemporary AI chips with modern processors. Source: <a href=\"https:\/\/cset.georgetown.edu\/\" target=\"_blank\" rel=\"noopener\" title=\"\">Center for Security and Emerging Technology<\/a>.<\/figcaption><\/figure>\n<h2 class=\"wp-block-heading\"><strong>Market potential\u00a0<\/strong><\/h2>\n<p>AMD chief executive Lisa Su <a href=\"https:\/\/in.marketscreener.com\/quote\/stock\/ADVANCED-MICRO-DEVICES-IN-19475876\/news\/Lisa-Su-CEO-of-AMD-The-AI-sector-could-reach-400-billion-by-2027-45856367\/\">put<\/a> the total addressable market for AI chips at $400bn. To put the figure in context, New Street Research <a href=\"https:\/\/seekingalpha.com\/news\/4057979-amd-gains-as-new-street-research-upgrades-on-ai-hopes\">analysed<\/a> the industry\u2019s current state.<\/p>\n<p>For now, NVIDIA dominates, with $38-39bn of spending, followed by Broadcom, which designs TPUs for Google. Despite rapid growth in AI accelerators, GPUs are expected to account for the bulk of market expansion by 2027.<\/p>\n<p>New Street Research forecasts substantial growth in AI chips over the next few years. By 2027 the market could reach a scale equivalent to roughly 10m servers housing 100m chips. That corresponds to about 10% of internet users employing AI for various purposes.<\/p>\n<p>The embedding of AI into everyday applications and workflows will encourage its use and propel the AI\u2011chip industry.<\/p>\n<p>Industry leaders recognise the sector\u2019s potential. According to media reports, OpenAI head Sam Altman has <a href=\"https:\/\/forklog.com\/en\/news\/openai-seeks-7-trillion-for-ai-chip-production-ambitions\">led<\/a> an effort to raise up to $7trn for semiconductor expansion to ensure future AI development has the necessary compute.<\/p>\n<p>The initiative may not succeed. But it shows how seriously technology visionaries take the AI race.<\/p>\n<h2 class=\"wp-block-heading\"><strong>The impact of COVID-19 and generative AI\u00a0<\/strong><\/h2>\n<p>At the start of the COVID-19 pandemic the semiconductor industry sank into a severe crisis as supply chains faltered. Every sector dependent on chips\u2014from carmaking to entertainment and consumer electronics\u2014was hit.<\/p>\n<p>By 2022 the situation began to stabilise. Leading manufacturers adapted to the new reality, and some pandemic restrictions were lifted. The launch of ChatGPT in late November 2022, however, heaped fresh pressure on a just-recovering industry.<\/p>\n<p>The staggering success of a service from a small start-up did not go unnoticed by tech giants. Microsoft, Google, Amazon, Meta and many others entered a new race to develop generative AI.<\/p>\n<p>The chip industry now risks another painful shortage. Demand for AI chips is extraordinary; producers are selling future batches months ahead, and a plethora of products is arriving from both established and lesser-known firms.<\/p>\n<h2 class=\"wp-block-heading\"><strong>The main players\u00a0<\/strong><\/h2>\n<p>Today NVIDIA leads in chips for AI training, with an 80% share. Since early 2023, the boom in generative AI has sent the firm\u2019s market capitalisation from $364bn to $2.19trn.<\/p>\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/lh7-eu.googleusercontent.com\/Xj1GNIn7VygXsgQjJKVbtaW5pK4rmMCsfmoqzI-_nhtNTt1s4i-Q3RCR_BupgxNn50U2nT_nIOh31tTqFCmo3G2GRhbs47RYdsdvAzoKyQirTbDnVE9ca6V1n_fVkgIRPogpOm4olpA3QSz9uTHnReo\" alt=\"\u0418\u0418-\u0447\u0438\u043f\u044b \u0432 2024 \u0433\u043e\u0434\u0443: \u043d\u0430 \u0447\u0442\u043e \u0421\u044d\u043c \u0410\u043b\u044c\u0442\u043c\u0430\u043d \u043f\u0440\u0435\u0434\u043b\u0430\u0433\u0430\u0435\u0442 \u0442\u0440\u0430\u0442\u0438\u0442\u044c \u0442\u0440\u0438\u043b\u043b\u0438\u043e\u043d\u044b\"\/><figcaption class=\"wp-element-caption\">NVIDIA\u2019s market capitalisation, 2001\u20132024. Source: <a href=\"https:\/\/companiesmarketcap.com\/nvidia\/marketcap\/\">CompaniesMarketCap<\/a>.<\/figcaption><\/figure>\n<p>Firms such as Intel and AMD are also investing billions of dollars in specialised AI chips as they seek a place in this promising market.<\/p>\n<p>Cloud providers including Microsoft, Amazon and Google are building their own AI chips. Though still NVIDIA customers, the tech giants want to reduce reliance on external suppliers and secure supply amid looming shortages.<\/p>\n<p>Other players are in the market too, mostly with narrower offerings. They include Kneron, Cerebras Systems and Lightmatter.<\/p>\n<p>Crucially, few foundries can manufacture at cutting-edge nodes. The companies above typically order semiconductors from TSMC, Intel and Samsung.<\/p>\n<h2 class=\"wp-block-heading\"><strong>What next<\/strong><\/h2>\n<p>The specialised AI\u2011chip industry is set for rapid development. Key drivers will be rising demand, adoption of advanced process nodes and large-scale investment in semiconductors. Leading players will focus on innovation.<\/p>\n<p>In 2024\u20132025 TSMC will <a href=\"https:\/\/www.tomshardware.com\/news\/tsmc-races-to-2nm-nvidia-apple\">start <\/a>shipping initial batches and small\u2011run production of 2nm chips. Mass production is expected in the second half of 2025.<\/p>\n<p>The move to 2nm targets energy efficiency. Transistor speed is expected to rise by 10\u201315% at the same power, or power consumption to fall by 20\u201330% without sacrificing performance.<\/p>\n<p>In 2026 TSMC will <a href=\"https:\/\/www.anandtech.com\/show\/17453\/tsmc-unveils-n2-nanosheets-bring-significant-benefits\">move<\/a> to a second\u2011generation 2nm N2P process with backside power delivery. After mastering 3nm and 1.4nm nodes, Intel <a href=\"https:\/\/www.anandtech.com\/show\/15217\/intels-manufacturing-roadmap-from-2019-to-2029\">plans<\/a> to develop a 1nm process, where 1nm is just four silicon atoms across.<\/p>\n<p>AI chips in consumer electronics are also set to advance. Manufacturers such as Intel, AMD, Apple and Qualcomm are adding specialised <a href=\"https:\/\/ru.wikipedia.org\/wiki\/%D0%9D%D0%B5%D0%B9%D1%80%D0%BE%D0%BD%D0%BD%D1%8B%D0%B9_%D0%BF%D1%80%D0%BE%D1%86%D0%B5%D1%81%D1%81%D0%BE%D1%80\">neural processing units<\/a> (NPUs) to their products. These will run AI tasks directly on users\u2019 devices.<\/p>\n<p>For example, Qualcomm\u2019s current Snapdragon 8 Gen 3 already <a href=\"https:\/\/www.theverge.com\/2023\/10\/24\/23928867\/qualcomm-snapdragon-8-gen-3-on-device-ai-meta-llama-2\">can<\/a> deploy models such as Stable Diffusion and Meta Llama 2. Developers say it can generate an image in under a second\u2014the previous technology took roughly 15 seconds. By comparison, a well\u2011equipped laptop may need up to two minutes to create a picture.<\/p>\n<p>In sum, continual improvements in manufacturing will yield ever smaller, faster and more energy\u2011efficient AI chips. Heavy investment in this promising sector will spur a wave of innovation.<\/p>\n<p><em>Text: Bohdan Kaminsky<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Artificial intelligence is already becoming integral to most spheres of activity. Alongside advances in algorithms, hardware\u2014especially specialised chips\u2014plays a decisive role. Modern AI methods depend on computations at a scale that was impossible only a few years ago. Training cutting-edge algorithms can take months and cost tens of millions of dollars. Such colossal compute is [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":11809,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"select":"","news_style_id":"","cryptorium_level":"","_short_excerpt_text":"","creation_source":"","_metatest_mainpost_news_update":false,"footnotes":""},"categories":[1144],"tags":[438,1295,1455],"class_list":["post-11810","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-longreads","tag-artificial-intelligence","tag-chips","tag-technology"],"aioseo_notices":[],"amp_enabled":true,"views":"33","promo_type":"","layout_type":"","short_excerpt":"","is_update":"","_links":{"self":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts\/11810","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/comments?post=11810"}],"version-history":[{"count":0,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/posts\/11810\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/media\/11809"}],"wp:attachment":[{"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/media?parent=11810"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/categories?post=11810"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/forklog.com\/en\/wp-json\/wp\/v2\/tags?post=11810"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}