Key AI Deployment Strategies from Top Agentic AI Leaders
▼ Summary
– VB Transform 2025 highlighted lessons from enterprise leaders on deploying AI agents, emphasizing the need for evaluation and orchestration infrastructure before scaling.
– ROI for AI agents extends beyond cost reduction, with examples like Rocket Companies saving $1M annually and Cognigy reducing call handling times.
– AI agents can generate revenue by improving customer conversion rates and enabling proactive outreach, as seen in food delivery and financial services.
– Many AI agents fail in production due to lacking evaluation infrastructure, requiring AI-specific testing methods like simulation to handle unpredictable scenarios.
– Enterprises must prepare for a future with hundreds of interconnected AI agents, as retrofitting infrastructure later will be more costly and complex.
Businesses racing to implement AI agents are discovering that success hinges on strategic infrastructure rather than just powerful models. At a recent industry event, top executives from leading companies revealed critical insights about deploying AI at scale, highlighting both the substantial rewards and hidden pitfalls of agentic systems.
The real ROI of AI agents extends far beyond cost savings. While early adopters focused primarily on reducing expenses, enterprises now see more nuanced returns that require tailored technical architectures. Shawn Malhotra, CTO of Rocket Companies, shared how a simple AI agent built in two days slashed $1 million annually in mortgage underwriting costs. Meanwhile, Cognigy’s Thys Waanders emphasized that automating call handling significantly lowers operational expenses.
But the financial upside doesn’t stop there. Proactive AI-driven engagement unlocks entirely new revenue streams. Malhotra noted faster query resolution boosts customer conversion rates, while Sendbird’s Shailesh Nalawadi highlighted how preemptive customer service, like alerting users about late food deliveries before complaints arise, strengthens brand loyalty and retention.
The biggest roadblock? Skipping evaluation frameworks. Many companies rush AI agents into production without proper testing, leading to costly failures. Nalawadi stressed that traditional software QA methods fall short for AI systems, which require dynamic evaluation tools akin to “unit tests for agents.” Without them, unpredictable edge cases, like slang or multilingual interactions, can derail deployments.
To tackle this, Cognigy developed AI-powered simulation testing, where synthetic agents stress-test real ones across thousands of scenarios. This uncovers behavioral flaws, from handling irate customers to processing regional dialects, issues human testers often miss.
Looking ahead, the next challenge is managing interconnected AI ecosystems. As enterprises scale from single-task bots to hundreds of collaborative agents, monitoring complexity skyrockets. Retrofitting systems later is prohibitively expensive, making forward-looking architecture essential.
The takeaway? Success demands balancing ambition with preparation. Companies that prioritize evaluation, simulation, and scalable infrastructure will thrive, while those fixated solely on model capabilities risk costly missteps.
(Source: VentureBeat)
