NeuReality taps ex-Google AI director as adviser

Last week, Jensen Huang described the future data center as a “token factory” to a massive audience at GTC. This vision aligns with the foundational work of NeuReality, an Israeli startup that has just strengthened its strategic direction. The company announced the appointment of Shalini Agarwal, a product management director from Google Labs, as a strategic adviser. Her focus will be guiding the market strategy for NeuReality’s flagship NR-NEXUS inference operating system.

This move marks an evolution for a firm that initially focused on custom AI inference silicon before pivoting to software. The company now aims to transform disparate GPU clusters into streamlined, production-ready inference engines. Agarwal brings about twenty years of product strategy experience from major tech firms, including her recent AI-focused work at Google Labs and a prior decade at eBay. An MIT graduate, her advisory role adds a notable Silicon Valley presence to the leadership team alongside CEO Moshe Tanach and President Hiren Majmudar.

The timing of this appointment is strategic. NeuReality publicly launched NR-NEXUS in mid-March, presenting it as a hardware-agnostic operating system for building AI factories. The platform’s core function is to disaggregate prefill and decode tasks across a mix of hardware, including GPUs, CPUs, and network interface cards. The goal is to maximize utilization and extract more productive work from expensive accelerators that frequently remain underused. The company reports that beta customers are already testing the software, though their identities remain confidential.

NeuReality is entering the market at a pivotal moment. Inference economics have become a critical focus in enterprise AI, with Deloitte estimating inference workloads will constitute two-thirds of all AI compute this year. Hyperscalers are committing colossal capital expenditures, yet the ecosystem remains dominated by a few vertically integrated stacks. This leaves many enterprises, seeking to run inference across heterogeneous hardware environments, with few practical solutions.

NR-NEXUS is designed to address this exact gap. The platform is built to operate across any CPU, GPU, or NIC, with support planned for NVIDIA’s upcoming Vera Rubin architecture. NeuReality is targeting three primary customer segments: neocloud providers, enterprises building private inference capacity, and semiconductor companies wanting to offer a complete software layer with their chips.

To fuel its ambitions, NeuReality has secured roughly $70 million in funding. A $35 million Series A round in late 2022 was followed by a $20 million investment in March 2024 led by the European Innovation Council Fund, framing the company as part of Europe’s push for sovereign AI infrastructure, despite its main engineering center being in Israel.

Agarwal’s advisory role underscores a key realization for the startup: engineering a superior inference operating system is only part of the battle. The other, equally crucial half is go-to-market strategy. Success depends on convincing infrastructure buyers, who often have entrenched relationships with established software ecosystems like NVIDIA’s, that integrating a new orchestration layer is a worthwhile endeavor.

The company faces significant, well-funded competition in a rapidly heating market. Rivals like Modal Labs, Baseten, and Fireworks AI are all pursuing the same fundamental opportunity from different angles. As AI shifts from training to widespread deployment, control over the inference layer is increasingly seen as controlling a major portion of the value chain.

On the surface, bringing on a Google-level product adviser is a tactical step. In reality, it represents a strategic wager on the next phase of AI infrastructure. NeuReality is betting that the market will reward companies which can effectively bridge the divide between advanced silicon and the enterprises that need to run models efficiently, at scale, on their existing hardware investments.