OpenAI unveils first custom chip, built with Broadcom
▼ Summary
– OpenAI unveiled Jalapeño, its first custom inference processor, designed with Broadcom and assisted by its own AI models.
– Early tests show Jalapeño offers significantly better performance-per-watt than current state-of-the-art alternatives.
– The chip aims to reduce OpenAI’s dependence on Nvidia GPUs, following similar custom chip efforts by Google and Amazon.
– Jalapeño is optimized for inference tasks, emphasizing low operating costs for real-time coding models, while pre-training likely still relies on Nvidia hardware.
– OpenAI designs the chip as part of a broader strategy to optimize infrastructure across chip architecture, kernels, memory, and networking to make models faster and more affordable.
On Wednesday, OpenAI introduced its first custom-built inference processor, developed in partnership with Broadcom. Dubbed Jalapeño, the chip was tailored specifically for the demands of OpenAI’s inference infrastructure. According to the company, its own AI models played a role in designing the processor.
Although testing is still underway, OpenAI reports that early benchmarks indicate significantly better performance-per-watt compared to current state-of-the-art options.
The collaboration was first made public in October, though rumors about OpenAI’s chip ambitions had circulated for months as a strategy to lessen reliance on Nvidia’s GPUs. Similar moves have been made by Google and Amazon, both of which have created custom chips known as AI accelerators , silicon engineered to accelerate machine learning tasks.
Shortly after the Broadcom deal was confirmed, OpenAI president Greg Brockman discussed the company’s chip development philosophy on its internal podcast.
“We have a deep understanding of the workload,” Brockman said. “We’ve really been looking for specific workloads that are underserved, [and asking] how can we build something that will be able to accelerate what’s possible?”
Jalapeño is built exclusively for inference, the process of executing pre-trained AI models in response to user inputs. In its announcement, OpenAI highlighted the chip’s low operating cost when handling real-time coding models. More resource-heavy tasks, such as pre-training, are likely to remain dependent on Nvidia hardware. Still, even modest reductions in inference expenses could meaningfully boost the company’s profitability.
Optimizing inference could become a pivotal factor in the economics of artificial intelligence moving forward , and that optimization is expected to happen across every layer of the technology stack. OpenAI is already developing agentic products like Codex, the models that power them, and the data centers needed to run them. By moving into custom chip design, the company gains even more control, as outlined in its announcement.
“OpenAI is not only developing frontier models or building products on top of them; it is designing the infrastructure underneath them: chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience,” the company wrote. “Because OpenAI operates across the stack, each layer can be optimized around the same goal: making its models faster, more reliable, and more affordable for users.”
(Source: TechCrunch)
