Nvidia: Bull Case Is Much More Powerful Than You May Think
Contributor since 2018 / 1365 Followers
Summary:
- Financial media outlets and investors keep parroting about the moat Nvidia has built around its AI chip through the CUDA software, but actually, Nvidia’s moat extends way beyond that.
- While competitors are trying to build competing AI chips and accompanying software packages, CEO Jensen Huang is focusing on the grander opportunity, Data Center as a product.
- Nvidia is benefitting from multiple network effects simultaneously across its suite of data center solutions.
Justin Sullivan
Amid the generative artificial intelligence (“AI”) frenzy, the investment community has been going wild over the Nvidia Corporation’s (NASDAQ:NVDA) dominant market share in the AI chip space. Amid rising competitors striving to build their own AI chips to try and eat into Nvidia’s market share, bulls emphasize that it is not just the superior performance of its A100 and H100 chips that help build Nvidia’s moat, but it is also the CUDA software package that is specifically optimized for Nvidia’s chips that solidify its moat.
However, this is just the tip of the iceberg. Nvidia benefits from multiple network effects across its suite of data center solutions. While competitors strive to build their own AI chips to compete with the tech giant, Nvidia is increasingly focused on “Data Center as a product,” bolstering the bull case for the stock.
Nvidia’s full stack problem, full stack challenge
Semiconductor companies that want to effectively compete with Nvidia would not only need to build a comparable AI chip, but also build out a commensurate software ecosystem around the chip that helps speed up the application development process for developers. Nvidia has indeed developed a strong ecosystem around CUDA (or Compute Unified Device Architecture), including a large developer community, third-party software and hardware vendors, and academic institutions. This ecosystem incurs a virtuous cycle whereby the more people use CUDA, the more third-party developers and other partners are incentivized to support it by writing even more programs for Nvidia’s GPUs, which in turn strengthens the ecosystem further.
Most investors are aware of this moat factor by now. However, this is just the tip of the iceberg, as Nvidia’s moat in the data center industry extends far beyond its superior AI chip and CUDA software package. While competitors are trying to build competing AI chips and accompanying software packages, CEO Jensen Huang is focusing on the grander opportunity, Data Center as a product.
On the Q1 2024 earnings call, CEO Jensen Huang mentioned:
It’s been known for some time, and you’ve heard me talk about it, that accelerated computing is a full stack problem but — is full stack challenged. But if you could successfully do it in a large number of application domains that’s taken us 15 years, it’s sufficient that almost the entire data centers’ major applications could be accelerated, you could reduce the amount of energy consumed and the amount of cost for our data center substantially by an order of magnitude. It costs a lot of money to do it because you have to do all the software and everything, and you have to build all the systems and so on and so forth. But we’ve been at it for 15 years.
Huang talks often about accelerated computing being a full stack problem and a full stack challenge, but what does he mean by that?
A “stack” is basically a collection of technologies, including the hardware components like GPUs and SmartNICs, and software components like libraries and frameworks, that work together to deliver the desired functionality.
When he says that accelerated computing is both a full stack problem and a full stack challenge, he is referring to the fact that it involves multiple layers of technology and that solutions are required at each of these levels both individually and aggregately.
So, when he says that accelerated computing is a full stack problem, he means that it requires solutions at every level of the computing stack. It’s not just about developing faster hardware, but it also involves optimizing software, designing programming frameworks that are able to optimally utilize the power of the different hardware components.
Then he simultaneously refers to it as a full stack challenge, whereby he is addressing the complexities incurred in maximizing the performance of accelerated computing holistically with each level of the stack working efficiently together. All these separate hardware components, and their accompanying software components, need to be effectively integrated together for optimal performance for the data center as a whole.
Networking solutions play a crucial role in integrating the various layers of the stack in accelerated computing, enabling efficient data transfer and communication between the various components within the data center infrastructure. As per Nvidia’s latest 10-K filing (emphasis added):
Networking solutions include InfiniBand and Ethernet network adapters and switches, related software, and cables. This has enabled us to architect end-to-end data center-scale computing platforms that can interconnect thousands of compute nodes with high-performance networking. While historically the server was the unit of computing, as AI and HPC workloads have become extremely large spanning thousands of compute nodes, the data center has become the new unit of computing, with networking as an integral part.
Amid the generative AI revolution, the InfiniBand networking technology (obtained through the Mellanox acquisition in 2020) that will be in ultra-high demand as data center operators re-design their data centers to be apt for the era of AI. As CEO Jensen Huang proclaimed on the call:
In networking, we saw strong demand at both CSPs and enterprise customers for generative AI and accelerated computing, which require high-performance networking like NVIDIA’s Mellanox networking platforms. … As generative AI applications grow in size and complexity, high-performance networks become essential for delivering accelerated computing at data center scale to meet the enormous demand of both training and inferencing. Our 400-gig Quantum-2 InfiniBand platform is the gold standard for AI-dedicated infrastructure, with broad adoption across major cloud and consumer internet platforms, such as Microsoft Azure.
…
InfiniBand had a record quarter. We’re going to have a giant record year. And InfiniBand has a really — NVIDIA’s Quantum InfiniBand has an exceptional roadmap. It’s going to be really incredible. But the two networks are very different. InfiniBand is designed for an AI factory, if you will. …The difference between InfiniBand and Ethernet could be 15%, 20% in overall throughput. And if you spent $500 million in an infrastructure and the difference is 10% to 20% and it’s a $100 million, InfiniBand is basically free. That’s the reason why people use it. InfiniBand is effectively free. The difference in data center throughput is just — it’s too great to ignore
Therefore, putting the superiority of its AI chip aside for a moment, networking technologies like InfiniBand also play a major role in sustaining Nvidia’s data center moat. Essentially, Nvidia’s InfiniBand integrates the various components of the data center so efficiently, delivering outstanding results in terms of rapid data transfer and low-latency connections, that the resulting improvement of 20% data center throughput (the amount of data that can be transferred or processed within a data center over a given period of time) dwarfs the cost of the InfiniBand.
This offers two advantages. Firstly, the incredible cost efficiencies of Nvidia’s data center solutions make it increasingly difficult for competitors to penetrate Nvidia’s market share. Secondly, as InfiniBand becomes better and better at delivering cost efficiencies in terms of data center throughput, it yields greater pricing power to Nvidia, as data center customers don’t mind paying higher prices for such solutions as long as it is easily covered by the resulting cost efficiencies.
Digging a bit deeper into networking, Nvidia offers various networking solutions to help optimize the performance of data centers. This includes the Smart Network Interface Cards (SmartNICs), which offload certain network-related tasks from the CPU, freeing up processing power for other tasks. SmartNICs are designed to improve network performance, reduce latency, and enhance overall data center efficiency. These are accompanied by the NVIDIA Mellanox Software Development Kit [SDK] and the Mellanox Messaging Accelerator (MMA) software. Developers can build upon MMA to optimize and enhance the performance of their applications in data center environments. MMA provides a foundation for efficient message passing and network communication, and developers can leverage its capabilities to further customize and tailor their applications according to their specific requirements.
Networking solutions also include Networking Switches, which are devices that serve as central points for connecting and directing network traffic between the devices, servers, storage systems, and other components within the data center. These are accompanied by the NVIDIA Cumulus Linux software. NVIDIA Cumulus Linux is a network operating system designed to run on open networking switches. It provides developers with a flexible and customizable platform to optimize and enhance the functionality of networking switches in data center environments.
In terms of other data center hardware solutions, Nvidia also offers the NVIDIA Bluefield Data Processing Units (DPUs), which are designed to accelerate a wide range of data processing tasks beyond what traditional CPUs and GPUs are capable of, including networking, storage, and security functions. In its latest annual report, Nvidia states (emphasis added):
The NVIDIA Bluefield DPU is supported by foundational data-center-infrastructure- on-a-chip software, or DOCA, that lets developers build software-defined, hardware-accelerated networking, security, storage and management applications for BlueField DPUs. Partners supporting Bluefield include many of the top security, storage and networking companies. We can optimize across the entire computing, networking and storage stack to deliver data center-scale computing solutions.
BlueField DPUs, and its accompanying software, enhance the comprehensiveness of Nvidia’s data center solutions, bolstering its competitiveness in the market.
Hence, while investors are hinged on the well-established CUDA software package that is optimized for Nvidia’s GPUs, Nvidia is also winning from other network effects occurring simultaneously thanks to other software such as DOCA and MMA that optimize the other components involved in building data centers. CEO Jensen Huang emphasized on the call:
So, nearly everybody who thinks about AI, they think about that chip, the accelerator chip and in fact, it misses the whole point nearly completely. And I’ve mentioned before that accelerated computing is about the stack, about the software and networking, remember, we announced a very early-on this networking stack called DOCA and we have the acceleration library call Magnum IO. These two pieces of software are some of the crown jewels of our company. Nobody ever talks about it, because it’s hard to understand, but it makes it possible for us to connect 10s of 1,000s of GPUs.
Integrating all the data center components together for optimal holistic performance is a key attribute of Nvidia’s data center success. Data center customers interested in Nvidia’s industry-leading AI chips would subsequently also buy Nvidia’s other data center solutions for optimized performance through the deep integration benefits. This is what allows Nvidia to offer such jaw-dropping sales forecasts of $11 billion for Q2 2023, as it’s not just demand for AI chips, but also robust demand for all the other data center solutions. As the integrations continue to deepen, Nvidia becomes increasingly better positioned to cross-sell adjacent data center solutions together.
Risk to Nvidia bull case
Advanced Micro Devices, Inc. (AMD) also offers a comprehensive suite of data center solutions, and is striving to catch up to Nvidia in the AI race. In Q1 2023, AMD saw nearly $1.3 billion in data center revenue, representing 24% of total revenue. In comparison, Nvidia’s latest quarterly report for Q1 2024 revealed that the company made nearly $4.3 billion in data center revenue, contributing 60% to company-wide revenue.
The recent announcement of AMD’s rival AI chip, the MI300X, was underwhelming given the lack of performance specifications offered. AMD also failed to announce major customer partnerships for its new chip, which is testament to just how strong of a foothold Nvidia has in this market.
Nonetheless, witnessing Nvidia race forward towards the grander opportunity of “data center as a product’, AMD has also been beefing up its own data center solutions by acquiring Pensando Systems, Inc. (acquisition completed in May 2022), which makes networking chips.
“Pensando’s distributed services platform will expand AMD’s data center product portfolio with a high-performance data processing unit (DPU) and software stack that are already deployed at scale across cloud and enterprise customers including Goldman Sachs, IBM Cloud, Microsoft Azure and Oracle Cloud.”
AMD could indeed aggressively step up its M&A strategy to catch up to Nvidia. Afterall, Nvidia built up its own data center networking solutions through acquiring companies like Mellanox and Cumulus Networks.
Given Nvidia’s existing dominance in the space, AMD could try to engage in price competition to lure data center customers into using its expanding suite of solutions. However, it will be difficult for AMD to challenge Nvidia’s dominance through price competition, given just how well Nvidia has mastered the architecture of data centers through driving deep integrations across data center infrastructure. And it is these intricate integration benefits, and the resulting cost savings, that ultimately underpin Nvidia’s data center moat, given Jensen Huang’s long-standing approach towards accelerated computing as a full stack problem, and a full stack challenge. Consequently, Nvidia is strongly positioned to counter competitive threats, as Jensen Huang affirmed:
We’re mindful of competition all the time, and we get competition all the time. But NVIDIA’s value proposition at the core is, we are the lowest cost solution. We’re the lowest TCO [Total Cost of Ownership] solution. And the reason for that is, because accelerated computing is two things that I talk about often, which is it’s a full stack problem, it’s a full stack challenge, you have to engineer all of the software and all the libraries and all the algorithms, integrated them into and optimize the frameworks and optimize it for the architecture of not just one chip but the architecture of an entire data center, all the way into the frameworks, all the way into the models.
AMD will inevitably also strive to drive down TCO for data center customers through deep integrations across its suite of solutions, but it will be an uphill battle. Nonetheless, AMD’s intensifying encroachment into Nvidia’s space could potentially undermine its pricing power over time, impacting Nvidia’s stock price, which is pricing in vigorous pricing power.
That being said, Nvidia is certainly not going to stand still while AMD sprints to catch up. Nvidia will continue to enhance the cost efficiencies and performance of its own solutions on an aggregate basis, to sustain its lead. On top of that, the multiple network effects running simultaneously across Nvidia’s key data center solutions (other than CUDA for its AI chip) erect strong ecosystems around its products, steepening the uphill battle for AMD.
Is Nvidia stock a buy?
Nvidia’s data center moat extends beyond its superior AI chip and CUDA software package. While competitors are trying to build competing AI chips and accompanying software packages, CEO Jensen Huang is focusing on the grander opportunity, Data Center as a product. The company’s deep integration of data center components, such as SmartNICs, DPUs, and networking switches, bolsters its competitiveness and cross-selling opportunities.
While AMD is expanding its data center solutions and could engage in price competition, Nvidia’s focus on full stack problem-solving and cost efficiency makes it difficult for competitors to challenge its dominance. Nvidia’s powerful moat solidifies the bull case for the stock, making it a “Buy.”
Analyst’s Disclosure: I/we have a beneficial long position in the shares of NVDA either through stock ownership, options, or other derivatives. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.
Seeking Alpha’s Disclosure: Past performance is no guarantee of future results. No recommendation or advice is being given as to whether any investment is suitable for a particular investor. Any views or opinions expressed above may not reflect those of Seeking Alpha as a whole. Seeking Alpha is not a licensed securities dealer, broker or US investment adviser or investment bank. Our analysts are third party authors that include both professional investors and individual investors who may not be licensed or certified by any institute or regulatory body.