AI infrastructure demand shows no signs of slowdown: New Street

Artificial intelligence infrastructure shows no signs of slowing down any time soon as higher spend on computing power leads to better performing models, according to New Street Research.

It is a like a self-perpetuating system.

“OpenAI showed how model size is turning into heavy pressure on inference costs,” said New Street Research analysts, led by Pierre Ferragu, in an industry report.

“Although inference costs come down very fast as models mature, next-generation models are significantly more expensive to run than the previous version (and this will only get worse with the introduction of reasoning capabilities),” he added.

Although this cycle guarantees high demand and sales for the companies providing the hardware powering AI, it raises questions as to whether the ever-increasing expense of next-level models are worth it.

“Generative AI is a business in which the incremental cost is horrendously expensive (the latest model),” Ferragu noted. “This is the exact opposite of software, in which the incremental cost is zero. We are not sure how this will pan out, but it will be profoundly disruptive for the structure of the value chain.”

Companies such as Meta Platforms (NASDAQ:META) and xAI, the startup founded by Tesla (TSLA) CEO Elon Musk, continue to up the ante in AI hardware race. Last week, Musk announced an AI training cluster powered by 100,000 of Nvidia’s H100 GPUs. He plans to double in the next few months. Similarly, Meta CEO Mark Zuckerberg plans to have 350,000 H100s powering his company’s AI endeavors by the end of the year.

“As we improve our understanding of the economics of AI, both on the supply side and the demand side, we see a strong case for the sustainability of current levels of spending and room for material further growth,” Ferragu said. “This doesn’t mean there won’t be a pause or a correction, but it means we don’t see ourselves at the peak of an unsustainable bubble yet. Far from it.”

Nvidia is currently winning the race to provide the processors necessary to power the ever-evolving array of AI models. A primary reason is the full-stack and system-scale integration abilities of Nvidia’s latest GPUs. This allows for greater chip-to-chip bandwidth interconnect across a larger domain.

New Street Research said this is what gives Nvidia an advantage over competitor AMD (NASDAQ:AMD).

“It nevertheless doesn’t disqualify AMD, whose architecture is less versatile, less scalable, but can still be competitive, but on a much more limited set of use-cases, and at the cost of dedicated optimization efforts,” Ferragu added.

AMD understands this though and is working to challenge Nvidia, which is one reason why AMD recently acquired ZT Systems.