China’s ByteDance boosts efforts for self-designed AI chips – report

TikTok owner ByteDance (BDNCE) is advancing efforts to make its own AI chips as it intends to get an edge over its competitors in China’s AI chatbot market, The Information reported.

ByteDance is planning for mass manufacturing of two semiconductors it has designed by 2026, in collaboration with Taiwan Semiconductor Manufacturing (TSM), the report added citing people familiar with the matter.

The move could reduce the Chinese company’s reliance on costly Nvidia (NVDA) chips for developing and operating AI models, the report added.

China’s cloud services market has seen a price war with players such as Baidu (BIDU) Alibaba (BABA), ByteDance (BDNCE), Tencent (OTCPK:TCEHY) (OTCPK:TCTZF) and iFlytek cutting prices of their large language models, or LLMs, which power AI chatbots.

Reducing chip costs is a need for ByteDance. Last year the company released its first AI chatbot Doubao which provides generative AI content of text- and image. This year, ByteDance announced a group of low-cost LLMs, some of which cost up to 99% less than comparable offerings from OpenAI for generating a token — a word or part of a word generated by the models. Microsoft (MSFT)-backed ChatGPT creator OpenAI does not provide its AI models in China.

ByteDance has been investing to develop generative AI models. This year, the company placed orders for over 200,000 Nvidia H20 chips, the most advanced chip Nvidia made for the Chinese market which are allowed to be sold under the U.S. export rules, the report noted.

This order cost over $2B and ByteDance is still waiting to take full delivery, as per the report.

Now, ByteDance intends to order several hundred thousand of its own training and inference chips from TSM. The cost of these internally designed chips would be billions of dollars less than purchasing chips from Nvidia, as per estimates from The Information.

However, the chips are still in the design phase, and ByteDance’s plans could change.

The chip designs for its 2026 semiconductors are based on the advanced 5-nanometer technology, which is only one generation behind TSM’s most cutting-edge tech. The 5 nm process, which Chinese chip makers have found difficult to replicate, is the same generation of technology Nvidia is using for its upcoming flagship AI chip system Blackwell, the report noted.

The H20 can achieve about 25% the performance of the Nvidia H100, which was Nvidia’s most advanced chip when it was released in 2022. The U.S. had restricted Nvidia from selling the A100 and more powerful successors, including the H100, in China, as part of Washington’s efforts aimed at hindering China’s access to advanced semiconductor technology used for AI.

The H20 remains in high demand as Chinese companies navigate U.S. export control curbs. As the computing power of a single H20 is low compared to the more common Nvidia chips used by U.S. cloud providers, the Chinese firms need to compensate by buying more of them to make a single cluster with the same level of computing power. This adds to their costs, according to the report.

The U.S. curbs apply to individual chips based on their performance, but clusters do not have such restrictions. One of the people familiar with ByteDance’s chips said there were other ways Chinese chip engineers could use to increase the performance of chips, such as boosting the connection between each chip in a cluster and improving the chip’s performance when accessing high-bandwidth computer memory. The U.S. restrictions do not restrict either of these methods.

ByteDance desires clusters powered by its own chips to get four times the computing performance of a single H100 chip for about the same cost. The H100 chips can cost over $30,000 each.

The chip efforts by ByteDance are in the footsteps of companies, including Alphabet’s (GOOG) (GOOGL) unit Google, Amazon (AMZN) and Microsoft, who in the past few years have been trying to reduce their dependence on Nvidia chips for developing and running AI models.

Some of these companies also offer their own chips as an alternative to Nvidia for customers using their cloud services, the report noted.

Chinese companies, such as Tencent, Alibaba and Baidu have been developing their own AI chips for years. Baidu is developing a new AI chip called Kunlun 3, for its generative AI products, the report added, citing a person familiar with the project.

The Baidu chip is at the final stage of the design phase and nearing the start of production at TSM, the report noted.

In July, it was reported that Nvidia was working on a new chip for China which has a special server design that would not violate U.S. export restrictions.

Meanwhile, Chinese tech giant Huawei Technologies is reportedly nearing to unveil a new chip to use for AI that would compete with Nvidia’s chips in the Chinese market.