China’s ByteDance Developing New AI Chips Like Those from Nvidia Partner Groq
The Takeaway
- ByteDance is developing new AI chip for inference, similar to Groq’s language processing units.
- ByteDance is partnering with InnoStar Semiconductor for memory integration.
- The new design could avoid the need for high-bandwidth memory chips whose export to China is heavily restricted by the U.S.
TikTok owner ByteDance is developing a new chip to run artificial intelligence models as part of an aggressive expansion of its homegrown AI infrastructure.
The new AI chip is intended to have a structure similar to U.S. chip designer Groq’s language processing units, which are built to run AI models at low cost, according to three people with knowledge of the project. ByteDance, which is one of China’s AI leaders, is also working closely with InnoStar Semiconductor on how to integrate that Chinese startup’s memory technology into the new AI chip ByteDance is developing, the people said.
InnoStar, in which ByteDance invested in 2024, is now raising about $400 million from investors including ByteDance and Yunfeng Capital, an investment firm co-founded by Alibaba’s Jack Ma, at a pre-money valuation of roughly $1.5 billion, according to one of the three people and a separate person with direct knowledge of the round.
ByteDance’s previously unreported chip design efforts and its collaboration with InnoStar highlight the Chinese tech giant’s ambitions to expand its footprint in AI infrastructure, not just by investing heavily in its own data centers but also through collaborations with hardware suppliers. ByteDance, best known for its short-video apps, also develops a wide range of AI models and operates China’s most popular AI chatbot app, Doubao.
ByteDance has been racing to secure its own AI silicon as Beijing pushes Chinese tech giants away from Nvidia and the U.S. government curbs exports of Nvidia chips to China. Since last year, the Chinese government has banned tech companies from buying the H20, Nvidia’s scaled-down chip for Chinese customers designed to comply with U.S. export controls. Beijing wants to reduce China’s dependence on U.S. technology and become more self-sufficient in key areas such as semiconductors.
ByteDance’s new AI chip, like Groq’s, is designed for inference, the work of running a trained AI model to generate responses to prompts. Inference accounts for the bulk of ByteDance’s AI computing needs. Around the world, the rapid rise of AI agents has triggered an exponential surge in inference workloads.
ByteDance accelerated its efforts earlier this year after Nvidia struck a $20 billion deal to license Groq’s technology, according to two of the three people with knowledge of the project. The Groq deal showed how the surging demand for AI inference is creating the need for more specialized chips.
ByteDance’s chip ambitions go back to at least 2022, when it began hiring semiconductor engineers. The company previously planned to produce its advanced AI processors at Taiwan Semiconductor Manufacturing Company, The Information reported in 2024. While that chip was never launched, ByteDance is now working on a new AI processor, code-named Ada-S, in addition to the Groq-like inference chip project, according to two people with direct knowledge of the project.
ByteDance is also working on a separate chip for video algorithms, which is used to process filters across its short video apps, according to two ByteDance employees.
AI Inference Boom
The work on ByteDance’s new Groq-like chip is still at a relatively early stage and it is unclear when it can move to the production phase and which chip manufacturer ByteDance could work with, the people with knowledge of the project said.
ByteDance is following a global trend. In addition to Nvidia’s licensing deal with Groq, other U.S. tech giants are building AI accelerator chips that specialize in different parts of inference to increase efficiency. Google is in talks with U.S. chip company Marvel on new inference chips with similar design to Groq’s language processing unit, or LPU, The Information reported last month.
Groq’s LPU stores an AI model’s accumulated knowledge directly on the chip in a large pool of fast memory called static random-access memory. Such SRAM can be read many times faster than high-bandwidth memory, or HBM, which sits beside other AI accelerators such as those from Nvidia and Google.
Integrating memory into AI accelerator chips can increase speed and reduce power consumption, but there is an additional benefit for Chinese companies such as ByteDance. Like Groq’s LPU, the new chip ByteDance is developing won’t use HBM chips, which also are tightly restricted by U.S. export controls, according to two of the three people with direct knowledge of the project.
ByteDance’s partner, Shanghai-based InnoStar, founded in 2019, specializes in resistive random-access memory or RRAM, a type of memory technology that stores data as changes in electrical resistance inside a cell. InnoStar designs its chips in China but manufactures them at TSMC’s mature node production lines, which are not restricted by the U.S. export controls.
But ByteDance’s effort to use InnoStar’s memory for AI computing could take time, the people with knowledge of the collaboration said. RRAM is an emerging technology and the startup is now moving from research prototypes to commercial production.