Report Highlights

Key Decisions This Report Supports

This report addresses four critical decisions facing technology executives, investors, and government policymakers in the AI chip market. The first is the procurement decision: whether to standardise on NVIDIA's GPU platform (maximum software compatibility, highest availability, premium pricing), diversify to AMD GPU or Intel Gaudi, or wait for hyperscaler custom silicon access through cloud API rather than data centre hardware procurement. The analysis framework: training workloads strongly favour NVIDIA CUDA ecosystem for performance and tooling compatibility; inference workloads have more viable alternatives where throughput-per-dollar is the primary metric. The second is the supply chain decision: given the Taiwan Semiconductor Manufacturing Company concentration of leading-edge AI chip production, how should companies assess geopolitical supply disruption risk and what hedging strategies are available? The third is the investment decision: at current valuation multiples, is NVIDIA's market position durable enough to justify continued premium versus AMD, Arm Holdings, and AI chip pure-plays? The fourth is the policy decision: how should governments evaluate AI chip export controls, domestic manufacturing incentives, and national AI computing infrastructure investments to preserve competitive advantage?

Industry Snapshot

The AI Chip (GPU/TPU) market was valued at approximately USD 68.4 billion in 2024 and is projected to reach approximately USD 424.6 billion by 2034, growing at a CAGR of 20.0%–24.4%. NVIDIA's data centre GPU segment alone generated approximately USD 47.5 billion in 2024 revenue — representing approximately 69% of the addressable market defined here. AMD's data centre GPU revenue (MI300X and MI300A) reached approximately USD 5 billion in 2024, growing from near-zero in 2022, demonstrating the fastest revenue ramp in semiconductor history for a new product family. Google's TPU v5 powers the majority of Google's own AI inference workloads and is available through Google Cloud, representing one of the most deployed but commercially reported AI accelerators. The market is growing at its fastest rate in semiconductor history — AI workload demand is growing faster than Moore's Law can supply performance improvement, driving unprecedented data centre capital expenditure from all major cloud providers.

The Forces Accelerating Demand Right Now

Generative AI model training and inference is the primary demand driver — and the demand is structurally expanding, not merely a cyclical wave. Each generation of frontier AI model requires approximately 10x the compute of its predecessor: GPT-3 required approximately 3×10²³ FLOPS; GPT-4 approximately 2×10²⁴ FLOPS; the models in development for 2026–2027 training runs are estimated at 10²⁵–10²⁶ FLOPS. This exponential scaling creates a sustained capital expenditure obligation for frontier AI companies and hyperscalers that is independent of current AI monetisation — Meta, Google, Microsoft, and Amazon each committed to USD 40–60 billion in 2024 AI infrastructure capital expenditure, with guidance for continued acceleration through 2026. The inference demand from deployed AI services — Google Search AI Overviews, Microsoft Copilot, ChatGPT, Claude — is growing at 15%–25% monthly as user adoption expands and use case complexity increases, creating a recurring GPU demand floor that training demand adds to rather than substitutes for.

What Is Holding This Market Back

TSMC capacity constraint is the single most immediate supply limitation. TSMC's 3nm and 4nm N3/N4 process nodes — which manufacture NVIDIA's H100/H200 GPUs and AMD's MI300 series — have lead times of 12–18 months at full allocation. NVIDIA has been TSMC's largest 4nm customer since 2023, and the combined demand from NVIDIA, AMD, Apple, and hyperscaler custom silicon for TSMC's leading-edge capacity is running ahead of TSMC's capacity expansion schedule. CoWoS (chip-on-wafer-on-substrate) advanced packaging — which combines multiple GPU dies and HBM memory — is an additional bottleneck beyond the logic die itself, with CoWoS capacity constraining H100 and H200 shipments throughout 2023–2024. TSMC's USD 65 billion Arizona and Japan fab investments will expand capacity but not before 2026–2028 at commercial production volumes.

The Investment Case: Bull, Bear, and What Decides It

The bull case is AI model scaling continuing to require 10x compute per generation, hyperscaler capex maintaining USD 150–200 billion annual AI infrastructure investment through 2030, and NVIDIA maintaining 65%–75% of data centre AI accelerator revenue through CUDA ecosystem inertia. Probability: 55%–65% for the NVIDIA dominance durability scenario. The bear case is hyperscaler custom AI silicon (Google, Amazon, Microsoft, Meta) migrating 40%–60% of inference workloads off NVIDIA GPUs by 2028 — reducing NVIDIA's total addressable market in the highest-volume workload category and compressing data centre GPU revenue growth below current analyst consensus. Leading indicator: the proportion of inference (vs training) AI workloads running on non-NVIDIA hardware at Google, Amazon, and Microsoft by end-2025, which will be disclosed in partial capital expenditure disclosures.

Where the Next USD Billion Is Being Built

The 3–5 year commercial opportunity is edge AI inference silicon — custom AI chips for smartphones, PCs, vehicles, and IoT devices that perform AI inference locally without cloud dependency. Apple's Neural Engine (A-series and M-series SoC), Qualcomm's Hexagon AI NPU, and MediaTek's APU (AI Processing Unit) are the leading edge AI chip implementations. The market for edge AI inference semiconductor IP and chips is growing at 25%–35% annually as AI capabilities are embedded in every consumer device category. The 5–10 year transformative opportunity is specialised neuromorphic computing for always-on AI sensing — chips that process sensory data using brain-inspired sparse computing architectures at milliwatt power consumption, enabling AI in battery-powered IoT devices, medical wearables, and autonomous sensors that current GPU-architecture AI chips cannot serve economically.

Market at a Glance

Regional Intelligence

North America holds approximately 52% of AI chip market revenue — concentrated in US hyperscaler data centres (Amazon, Microsoft, Google, Meta) that are the primary purchasers of NVIDIA data centre GPUs, AMD AI accelerators, and their own custom silicon. NVIDIA, AMD, and Intel are all headquartered in the US, concentrating the AI chip design ecosystem despite TSMC Taiwan manufacturing dependency. Export controls — NVIDIA's H100/H200 and AMD MI300 are restricted from sale to China under BIS Entity List and CCL restrictions — have created a bifurcated global market that has accelerated China's domestic AI chip development. Asia Pacific holds approximately 28% — China's Huawei Ascend 910B (competitive with NVIDIA A100 in some benchmarks) and Biren Technology's BR100 GPU represent the domestic AI chip capability developed in response to US export controls; South Korea's Samsung and SK Hynix supply HBM (High Bandwidth Memory) that is critical to AI GPU performance; Taiwan's TSMC manufactures essentially all leading-edge AI chips regardless of design nationality. Europe holds approximately 12%, with no European AI chip design company at commercial scale — a gap that EU's Chips Act is attempting to address through fab investment incentives.

Leading Market Participants

• NVIDIA Corporation (H100, H200, Blackwell B100/B200 GPUs)
• AMD (MI300X/MI300A AI accelerators)
• Google (TPU v5 — internal and cloud)
• Amazon Web Services (Trainium2, Inferentia2)
• Intel (Gaudi3 AI accelerator)
• Microsoft (Maia 100 custom AI chip)
• Meta (MTIA custom inference chip)
• Qualcomm (Cloud AI 100, edge NPU)
• Arm Holdings (AI IP licensing for NPU)
• Huawei (Ascend 910B — China market)

Frequently Asked Questions

Q1. Frequently Asked Questions ▼

Q2. Why are GPUs better than CPUs for AI workloads? ▼

GPUs contain thousands of smaller, parallel processing cores optimised for the matrix multiplication operations that underpin neural network training and inference. An NVIDIA H100 GPU contains approximately 16,896 CUDA cores capable of executing multiple operations simultaneously. A high-end CPU contains 16–64 larger, general-purpose cores optimised for sequential instruction execution. AI model training requires billions of matrix multiply-accumulate operations in parallel — a task where GPUs achieve 100–1000x throughput versus CPUs performing the same computation serially. CPUs remain superior for sequential logic, system management, and workloads with complex branch prediction requirements that GPUs cannot parallelise efficiently.

Q3. What is CUDA and why does it create such durable competitive advantage for NVIDIA? ▼

CUDA (Compute Unified Device Architecture) is NVIDIA's parallel computing platform and programming model — a software layer that enables developers to write programs that execute on NVIDIA GPUs using standard programming languages. Since 2007, virtually all AI and deep learning frameworks (PyTorch, TensorFlow, JAX) have been built with CUDA as the primary GPU backend. Hundreds of thousands of developers have built CUDA-optimised code, libraries, and tooling; NVIDIA maintains cuDNN (neural network primitives), cuBLAS (linear algebra), and RAPIDS (data science) as highly optimised CUDA libraries that outperform generic implementations on competitive hardware. This software ecosystem — accumulated over 17 years — creates a switching cost for AMD and Intel that is not a hardware performance gap but a developer experience and tooling gap: models run on AMD ROCm or Intel oneAPI require porting effort and frequently achieve lower performance on the same mathematical hardware due to less optimised software stacks.

Q4. What is HBM (High Bandwidth Memory) and why is it critical for AI chips? ▼

High Bandwidth Memory (HBM) is a 3D-stacked memory technology that provides dramatically higher memory bandwidth than conventional GDDR memory — NVIDIA's H100 SXM achieves 3.35 TB/s of memory bandwidth using HBM3 memory, compared to approximately 0.8 TB/s for GDDR6X-based GPUs. AI model training and inference are frequently memory bandwidth-limited — the speed at which model weights and activations can be read from and written to memory, rather than raw compute throughput, determines overall performance. HBM's high bandwidth enables GPUs to maintain high computational utilisation even for large model sizes. Samsung and SK Hynix are the primary HBM suppliers — creating a supply chain dependency on South Korean manufacturers that is a separate concentration risk from TSMC's logic manufacturing dependency.

Q5. What are hyperscaler custom AI chips and why are they a threat to NVIDIA? ▼

Hyperscalers (Google, Amazon, Microsoft, Meta) are designing their own custom AI accelerator chips — Google's TPU, Amazon's Trainium and Inferentia, Microsoft's Maia, Meta's MTIA — optimised for their own AI model architectures and deployment workloads. The economic incentive is significant: a hyperscaler spending USD 10 billion annually on NVIDIA GPUs for inference workloads can potentially design a custom chip that achieves 2–3x better throughput-per-dollar for their specific use case, saving USD 3–5 billion annually at the cost of USD 500–800 million in chip design and tape-out investment. Custom chips are well-suited to inference (predictable, high-volume repetition of specific model architectures) but less suited to training (requires rapid model architecture changes during R&D that general-purpose GPUs accommodate more flexibly). The threat to NVIDIA is therefore concentrated in the inference segment as hyperscaler AI services scale to mass-market volumes.

Q6. How are US export controls affecting the global AI chip market? ▼

US Bureau of Industry and Security (BIS) export controls restrict NVIDIA's most powerful AI chips — H100, H200, and Blackwell B100/B200 — from export to China under both Entity List restrictions and aggregate compute threshold (FLOPS) controls. These restrictions have created a bifurcated global AI chip market: the US/allied countries market served by NVIDIA and AMD at full performance, and the China market where NVIDIA offers downgraded A800/H800 variants (below BIS thresholds) and domestic Chinese AI chips (Huawei Ascend, Biren, Cambricon) compete. China accounted for approximately USD 10–12 billion of NVIDIA's data centre GPU revenue in 2022 before restrictions intensified; tightening controls through 2023–2024 have reduced China's direct access to US leading-edge AI chips. The secondary effect — accelerating China's domestic AI chip development — is creating a long-term competitive dynamic that export controls cannot prevent, only delay.