Nvidia partners with chip rival d-Matrix on inference
▼ Summary
– Nvidia is partnering with AI chip startup d-Matrix to combine its GPUs with d-Matrix’s Corsair inference accelerator, with the first joint system set to be delivered to cloud firm Parasail later this year.
– The partnership leverages the two phases of AI inference: Nvidia’s GPUs handle the heavy “prefill” step, while d-Matrix’s specialized chips efficiently run the lighter “decode” step.
– d-Matrix’s Corsair chip uses in-memory computing to keep data near processing logic, avoiding expensive high-bandwidth memory, and claims it runs tokens 10 times faster at a third of the cost and up to five times less energy.
– This deal is part of Nvidia’s strategy to partner with rivals, allowing it to profit from the rise of specialized inference chips rather than lose market share if they take off.
– The collaboration signals a maturing AI-chip market where buyers seek the cheapest chip for each task, and Nvidia aims to remain central by selling or sitting alongside whatever solution wins.
Nvidia has found an unusual strategy for dealing with its chip competitors: partnering with them. The GPU powerhouse is teaming up with startup d-Matrix to integrate its hardware with the smaller company’s inference chips, according to The Information. The two firms will ship a joint system designed to run AI models, with the AI-cloud provider Parasail serving as its debut customer. That system is expected to go live later this year. The move fits a broader pattern for Nvidia, which increasingly collaborates with the very companies trying to challenge its dominance.
The logic behind the deal mirrors how AI workloads actually function. Answering a single prompt involves two distinct phases. The heavy “prefill” stage is a natural fit for Nvidia’s GPUs, while the lighter, repetitive “decode” stage can run more efficiently on specialized chips. By pairing the two, each component handles the task it performs best.
What d-Matrix brings to the table
d-Matrix built its Corsair accelerator around in-memory computing, a design that keeps data close to the processing logic. Founded in 2019, the startup raised $275 million at a $2 billion valuation last November and is now in the middle of another funding round. CEO Sid Sheth explains that the company’s advantage lies in placing both compute and memory on a single chip. It also avoids the expensive, scarce high-bandwidth memory that Nvidia’s chips rely on. According to d-Matrix, pairing Corsair with GPUs can process tokens about 10 times faster, at roughly one-third the cost, and with up to five times less energy on certain tasks.
Those claims come from the company itself, not independent benchmarks. Still, they point to where the real opportunity in AI now sits. Training built the models. Inference, running them for millions of users, is where the ongoing costs and the revenue increasingly converge.
Nvidia’s frenemy playbook
This partnership stands out because Nvidia rarely needs help. It dominates the AI chip market, and startups like d-Matrix exist specifically to erode that lead. Rather than fight every challenger, Nvidia keeps folding rivals into its ecosystem. It has used the same approach with other chip and networking companies.
The strategy hedges Nvidia’s bets. If specialized inference chips take off, similar to the custom silicon now spreading across big tech, Nvidia profits from the shift instead of losing ground. If they stall, the company loses little. Either way, customers remain inside Nvidia’s orbit.
Why it matters
The deal signals that the AI chip market is maturing beyond a single winner. Buyers no longer want one giant GPU for everything. They want the cheapest chip for each job, stitched together. Nvidia’s answer is to make sure it sells, or sits beside, whatever wins.
(Source: The Next Web)
