The Evolution of Computing Chips Through the Lens of Moore's Law | Source Code Capital Insights

源码资本·April 21, 2021·1·2

In the "post-Moore's Law" era, which direction will computing chip performance breakthroughs come from?

Weihua Cao joined Source Code Capital in 2020, focusing on investments in enterprise services and frontier technology. He previously worked at ByteDance's Strategy and Investment Department, participating in investments and acquisitions of multiple companies. Before that, he worked at Allston Trading in Chicago as a strategy analyst for U.S. equities and bond markets. Weihua holds a bachelor's degree from Tsinghua University and master's degrees from CMU and UC Berkeley.

Contact: cwh@sourcecodecap.com

Moore's Law, a classic principle governing the semiconductor industry's development, reveals how the exponential growth of integrated circuit computing power has had a disruptive, cascading effect across the entire supply chain. But as advanced process nodes demand massive capital investment, the returns from sustaining Moore's Law have steadily narrowed, and manufacturers have deliberately slowed the pace of technological iteration.

This article starts from the essential core of Moore's Law to consider how to strike a dynamic balance between technology development and product applications in the "post-Moore's Law era." The growth of IoT, cloud computing, big data, and AI has created new incremental application scenarios for the entire semiconductor supply chain, but the computing demands of different scenarios have also imposed more concrete requirements on technology. Moreover, China's chip industry has encountered a degree of tailwinds against this backdrop.

Source Code Capital presents this exclusive 22nd issue of Source Code Insights following analysis and research.

Moore's Law is an empirical rule within the semiconductor industry, not a law of the objective physical world.
Sustaining Moore's Law requires balancing capital R&D investment against the returns from advanced process nodes.
Increasing AI computing power and scenario-optimized integrated design thinking are driving continuous progress in computing chip performance.
The gradual失效 of Moore's Law is, in a sense, giving China's chip industry a chance to catch up.

The Origin of Moore's Law

The semiconductor industry has a well-known principle — Moore's Law, first proposed in 1965 by Gordon Moore, then chairman emeritus of Intel. At the time, Moore was preparing a report on trends in computer memory. While plotting the data for his summary, he discovered a striking pattern: each new chip roughly contained double the capacity of its predecessor, and each chip emerged 18 to 24 months after the previous one.

The famous "Moore's Law" was born. Its precise formulation: When price remains constant, the number of transistors that can be accommodated on an integrated circuit approximately doubles every 18 months, with performance also doubling. Put simply, every 18 months, the same amount of money buys four times the computing power.

To be precise, Moore's Law is an empirical rule within the semiconductor industry, not a law of the objective physical world like "Newton's Three Laws of Motion." What it reveals is the exponential growth of integrated circuit computing power.

Before 2005, the development speed of Intel CPU processors largely followed Moore's Law:

Image source: Internet

Advanced Process Development Entails Massive Capital Expenditure

The Gradual失效 of Moore's Law

In the semiconductor industry, the continuous iteration of process technology is a crucial underlying driver for maintaining Moore's Law. When we speak of semiconductor processes in terms of nanometers, we're referring to line width — the width of the most basic functional unit on a chip, the gate circuit. Since the width of interconnects between gate circuits is actually the same as the gate circuit width, line width can describe the manufacturing process. Currently, advanced processes have reached 5nm.

Advanced processes bring two benefits: 1) Shrinking line width means transistors can be made smaller and denser, and for the same chip complexity, smaller wafers can be used, thus reducing cost; 2) Shrinking line width can increase operating frequency. After reducing the distance between components, capacitance between transistors also decreases, allowing transistor switching frequency to rise, thereby increasing the overall chip operating frequency.

Continuous progress in advanced process technology:

But the advancement of advanced processes depends on continuous investment by foundries, requiring massive capital to build facilities and purchase the most advanced process equipment. Taking a fab capable of mass-producing 14nm process, 300mm wafers as an example, total investment is approximately $10 billion, with 65% of funds going to wafer manufacturing-related equipment such as lithography machines, etching equipment, physical vapor deposition (PVD) equipment, chemical vapor deposition (CVD) equipment, and other wafer manufacturing-related equipment.

Massive expenditure on fab construction:

Globally, leading foundries invest over $10 billion annually in capital and R&D. TSMC, as the leading pure-play foundry, spent $15.5 billion on capital and R&D in 2019. Intel and Samsung, representing the IDM model (integrating design, manufacturing, and packaging/testing), have even larger chip design teams; their capital expenditures plus R&D investments reached $29.5 billion and $36.4 billion respectively in 2019.

Capital and R&D investment by leading foundries far exceeds that of pure design companies (Fabless model):

Image source: Internet

Precisely because keeping pace with Moore's Law requires continuous capital expenditure, each generational advance in process technology means some players fall behind. Currently, only TSMC, Intel, and Samsung have mastered process technology at 7nm and below. SMIC has overcome the 10nm technology node and is further closing the gap with leading players.

Fewer and fewer players can keep up with advanced processes:

Image source: Internet

Even among the top 3 players, only TSMC's advanced processes are the most mature and stable. Intel has already begun falling behind — its development from 14nm to 10nm took five years (2014–2019). Starting from 14nm, Intel's price per unit area began rising exponentially. To control the price per transistor, Intel aggressively increased transistor density, leading to serious yield issues. Its 10nm advanced process still faces yield ramp and supply shortage problems, effectively raising product costs.

Intel's aggressive increase in transistor density:

Image source: Internet

For leading foundries, on one hand, sustaining Moore's Law means annual capital and R&D investments exceeding $10 billion — a very high cost burden. On the other hand, most applications of advanced processes are in consumer electronics, such as high-end flagship phones or laptops, which together account for only about 25% of the total semiconductor market. As global smartphone and laptop shipments have peaked and begun declining, the benefits of advanced processes are diminishing.

Insufficient demand growth for advanced processes:

Image source: Internet

Thus, the returns from sustaining Moore's Law are limited, and leading foundries have begun slowing the rate of advanced process iteration, seeking a balance between capital R&D investment and the returns from advanced processes — Moore's Law has失效.

As foundries slow investment in advanced processes, Moore's Law gradually失效:

Image source: Internet

Directions for Computing Chips in the "Post-Moore's Law Era"

From the analysis in the previous section, we can see that "Moore's Law" is first an economic law, and only then an engineering science law. When foundries sustain Moore's Law, bringing doubled chip integration and halved costs, from the system integrator's perspective this means doubled performance and doubled device integration. What users experience is that products in hand become increasingly usable and powerful — phones evolved from brick phones to feature phones to smartphones. Users can obtain more productivity or better user experience than their investment in processors would suggest by purchasing more advanced processors; processor manufacturers then invest their earnings in R&D to produce even more advanced processors; more advanced processors attract more users to buy, and so on. As the industry developed, the time for each advanced process generation remained around 18 months: if too slow, markets saturate and consumers lose motivation to upgrade, weakening the profitability of chip makers and foundries; if too fast, technological advances cannot surpass the accumulated gains of the previous generation, failing to maximize the previous generation's profitability and causing wasted investment.

The business logic embedded in Moore's Law:

If the cost of sustaining Moore's Law remained relatively constant, or if continuously more new users paid to amortize chip R&D and production costs, the business logic behind Moore's Law could continue operating indefinitely. But as penetration rates for smartphones and other smart terminals gradually saturate, and the investment costs required for advanced processes increase exponentially, sustaining Moore's Law has become increasingly unprofitable for chip vendors and foundries.

However, the development of new technologies such as IoT, cloud computing, big data, and AI has not weakened demand for data processing — especially AI, which requires processing massive amounts of image and text data. To address the mismatch between data growth and computing power growth, chip designers have begun seeking new approaches, with two representative directions being: 1) designing efficient parallel computing architectures for AI algorithms; 2) optimizing integrated hardware-software solutions for specific application scenarios.

Looking first at AI chips: currently mainstream AI algorithms are primarily deep learning algorithms, whose underlying operators involve massive parallel computing. Different scenarios impose different requirements on AI chips:

Image source: Internet

Compared to CPUs for general-purpose computing, AI chips are specialists, excelling at parallel multiply-accumulate operations. To use an analogy: a CPU is more like a university professor capable of handling various complex problems, while an AI chip is 100 elementary school students — unable to accept complex instructions, but in simple operations like multiplication and addition, the combined computing efficiency of 100 elementary school students far exceeds that of one professor. Thus, the design philosophy of AI chips is to increase parallel multiply-accumulate operation efficiency through architectural innovation, rather than relying solely on process iteration to boost computing power.

The main problem AI chips solve is how to efficiently process as many multiply-accumulate operations as possible per unit time. A "good" AI chip needs to excel in the following dimensions:

High throughput: Throughput = model QPS (queries per second), the number of model inferences completed per unit time; higher means more inferences per unit time.
Low latency: The time difference between model output and model input; lower is better.
High compute resource utilization: Refers to, given the same computing units, how many effective computing units are activated when running AI models. The number of activated computing units relates to the chip's instruction set design and resource scheduling system; higher utilization of the same computing resources is better.
Efficient inter-chip interconnect: AI models in real-world application scenarios often need to run across multiple chips, so for AI training chips it's not enough to stay at the single-chip level — efficient inter-chip interconnect protocols are also needed.
High flexibility: Whether there is comprehensive deep learning operator support; the more AI models that can run, the better.
Ease of use: No need to retrain or quantize; directly run already-trained models.

The second approach is to optimize hardware-software integrated solutions for application scenarios, thereby maximizing hardware potential to the greatest extent. Typical cases include Apple's newly released laptops and Amazon AWS servers.

Apple's latest 2020 laptop release features its self-developed M1 CPU. In addition to applying the most advanced process technology (5nm) and architectural innovations such as packaging memory dies onto the chip, unlike previous Intel CPUs, Apple's self-developed CPU is deeply customized and optimized for its own iOS, delivering stronger performance when running programs and correspondingly smoother user experience.

In 2002, Amazon's AWS launched cloud services powered by its self-developed Graviton CPU. Amazon EC2 T4g, M6g (general purpose), C6g (compute-optimized), and R6g (memory-optimized) instances based on Graviton2 provide up to 40% better price-performance for various workloads. The ARM-based reduced instruction set Graviton CPU, compared to Intel's x86 complex instruction set CPU, is better suited for heterogeneous computing.

What Moore's Law Teaches Us

Fifty-six years have passed since the proposal of "Moore's Law" in 1965. Looking back from today's vantage point, we find that Moore's Law can no longer accurately describe the iteration speed of advanced semiconductor processes. The economic principle underlying this is: the cost required to maintain advanced process iteration has gradually exceeded the value it brings, so the slowing of advanced process iteration speed follows naturally.

Yet on the other hand, the slowing development of advanced process technology does not mean slowing development of computing chip performance. We still see design approaches such as increasing AI computing power and vertically integrated optimization for specific scenarios driving continuous progress in computing chip performance, bringing improvements in user experience across various dimensions.

For China's chip industry, the gradual失效 of Moore's Law is, in a sense, a positive development — it means the slowing pace of leading international players, giving China's chip industry a chance to catch up. In fact, the main application domain of advanced processes is consumer electronics; 28nm/40nm process technology and 300mm wafers can already satisfy the vast majority of military and industrial scenario needs. As domestic 28nm lithography machines and other key chip manufacturing equipment gradually come online in the next 1–2 years, China will achieve supply chain autonomy in mature process technology.