Why 90% of Amazon’s Early AI Agents Never Shipped

The high failure rate of enterprise AI agents is a stark reality for technology leaders, with AWS vice president of agentic AI Swami Sivasubramanian revealing that nearly 90% of the AI agent prototypes developed by Amazon teams two years ago never made it to production. Speaking at the HumanX conference in Amsterdam, Sivasubramanian highlighted a broader industry struggle: while organizations are eager to adopt agentic workflows, few have successfully navigated the path from concept to deployment.

According to analyst data cited during the keynote, only 17% of organizations have managed to deploy AI agents effectively, and a mere 7% can accurately measure the return on investment (ROI). This gap creates significant pressure on engineering leaders. CTOs frequently report that their boards are demanding concrete metrics on these investments, yet many teams remain stuck in the experimental phase without clear business justification.

The Five Root Causes of Stalled Projects

To understand why so many initiatives fail, AWS deployed forward engineers to customer sites over a six-month period to diagnose systemic issues. The investigation identified five primary reasons projects stall. First, teams often work on the wrong problem because they lack a defined business outcome. Without clear goals, teams continue refining proof-of-concept models indefinitely, burning through tokens and budget without delivering value. Sivasubramanian compared this endless iteration to circling a roundabout in Paris multiple times on a first drive abroad.

Second, governance is frequently treated as an afterthought rather than a foundational element. Third, organizational structures are rarely redesigned to support AI workflows. Fourth, leadership paralysis sets in as executives oscillate between the fear of missing out and the fear of committing resources. Finally, teams struggle to measure whether their solutions actually work. OpenAI’s Colin Jarvis echoed these sentiments at the same conference, noting that enterprise AI is currently bottlenecked by deployment challenges rather than model capabilities.

Scaling Success Through Internal Innovation

Amazon’s strategy to overcome these hurdles involved allowing internal innovation to flourish while rigorously measuring outcomes. More than 100,000 Amazon engineers now use Kiro, the company’s agentic coding tool, alongside millions of external developers. A prime example of this approach is Bolin Chen, an engineer in the personalization team for Amazon Stores. Chen built an always-on assistant called MeshClaw using Kiro during his nights and weekends. Initially a solo project, it expanded rapidly after he shared it on an internal Slack channel, growing from one user to four founders and eventually thousands of contributors across the company.

When Amazon’s measurement systems flagged MeshClaw’s success, the Kiro team, which was already prototyping similar features like persistent memory and multi-agent coordination, merged forces with Chen’s group. This collaboration resulted in Kiro Crew, an open-source project that saw nearly 39,000 Amazon employees build upon it within 30 days of its internal launch. It also attracted 500 external contributors. The system even demonstrated self-improvement capabilities; an internal mechanism detected that the agent was redundantly reading unchanged files. The agent diagnosed the issue, wrote the fix, and validated it, reducing waste by 83%.

“No human diagnosed the problem. The system itself found it,” Sivasubramanian said.

Standardizing Infrastructure and Governance

Previously, Amazon teams had built agents across various services including EC2, EKS, and SageMaker, which created security and infrastructure burdens. To streamline operations, AWS consolidated its efforts on Bedrock for AI inference and AgentCore for agent hosting. This single, security-approved path simplifies compliance and reduces complexity.

A critical component of this standardization is Strands, AWS’s open-source agent framework, which introduces a feature known as putting an agent in a box. This involves a deterministic layer outside the agent that strictly governs which tool calls it can execute. As the agent proves its reliability, the boundaries can expand. This approach ensures safety and control.

“It’s not probabilistic. It’s always deterministic and mathematically provable that the agent cannot exceed its boundaries,” Sivasubramanian said.

Google DeepMind’s Kareem Ayoub also addressed the conference, suggesting that companies can effectively fence AI in to maintain control. This deterministic governance allows organizations to innovate safely without risking uncontrolled behavior.

Speed and Leadership Engagement

The effectiveness of this standardized approach is evident in AWS’s rapid scaling. Nearly 80% of the Fortune 100 now use Bedrock, making it the fastest-growing AWS service. When demand surged, a small team of six developers re-engineered the platform in just 76 days. Sivasubramanian noted that this work would have traditionally required 30 developers working for 18 months. The team eventually grew to 35 developers to handle the scaled load.

Leadership engagement remains crucial for maintaining momentum. Sivasubramanian shared an example where a scientist built the Amazon Quick desktop app overnight using Kiro and presented it to executives within a week. AWS launched the app externally within three months, and it is now used by hundreds of thousands of employees internally.

For organizations looking to replicate this success, Sivasubramanian advised leaders to ask two fundamental questions about every AI project: What outcome is it connected to, and is it compounding at every layer? High-ambiguity projects should start with very small teams to ensure agility and clear accountability. By focusing on defined outcomes, robust governance, and rapid iteration, companies can move beyond the prototype stage and deliver measurable value.