OpenID Foundation’s Plan to Tame Dangerous AI Agents

The rapid adoption of AI agents introduces significant new security vulnerabilities, exposing sensitive business data and critical processes to potential compromise. A recent technical analysis from the OpenID Foundation (OIDF) outlines these emerging dangers, arguing that without industry-wide cooperation on new, open identity and access management standards designed specifically for AI, these autonomous systems could easily bypass digital security barriers once considered secure.

The research focuses on the difficult balance organizations must strike between leveraging AI agents for productivity and governing their access to both internal and external data sources and computational services. Consider a scenario where an employee, seeking efficiency, grants an AI agent access to their email to automate customer responses. Currently, this might involve only a handful of staff, with risks managed through informal methods. However, in a few years, every employee could be using multiple agents, some of which might have been given unrestricted access to sensitive corporate resources without the IT department’s knowledge. The situation becomes even more precarious if these agents begin granting access to other agents, creating a complex web of permissions.

This could lead to a future where AI agents, each with broad, human-like access to corporate systems, vastly outnumber human employees. Relying on these agents to always act responsibly is not a viable security strategy. The OIDF’s work aims to alert stakeholders to the current state of AI identity management and the critical technical gaps that need urgent attention.

A major factor complicating this challenge is the Model Context Protocol (MCP), a capability that allows AI to dynamically adapt by connecting to a wide and growing range of data sources and services. While MCP is responsible for much of the “magic” behind advanced AI agents, it also presents a serious security dilemma. The protocol provides a standardized way for agents to discover and interact with virtually any resource, from structured data via APIs and unstructured document stores to other computational services and AI models.

In theory, the more resources that support MCP, the more capable and intelligent AI agents become. On the flip side, this also makes them more autonomous and less predictable. IT managers, who prefer known variables and predictable outcomes for risk assessment, are now faced with systems that do not behave like traditional software. AI agents take independent actions on external services, exhibiting flexible, non-deterministic behavior that adapts in real-time rather than simply following a predetermined script.

Although some initial steps have been taken to integrate identity and access management controls into MCP, these are currently insufficient for comfortably managing the autonomous nature of AI agents. A seemingly benign agent could easily conceal dormant malicious intent. As the paper’s author, Tobin South, noted, “MCP is definitely a double-edged sword. It opens up a ton of possibilities for AI agents but also introduces significant challenges for IT managers in terms of policy setting and control… Its current identity and authorization framework still needs work to robustly scale.”

The OIDF research proposes immediate improvements, chief among them is the concept of granting AI agents a “first-class” identity status similar to that of human users. This means applying the same foundational IAM controls to agents as to people. However, these controls must also be tailored with an awareness that the “user” is an AI, requiring specialized guardrails. These guardrails are essential for preventing unintended behaviors, mitigating risks, and maintaining trust by ensuring agents act responsibly and in alignment with human values.

These mechanisms represent a critical evolution of traditional Identity Governance and Administration principles. While a mature IGA program determines who can access what, AI guardrails provide a specialized, real-time control layer focused on how an agent uses that access. For example, IGA might grant an agent permission to a customer database, but an AI guardrail would enforce policies at the moment of action, such as automatically redacting Personally Identifiable Information before the data is sent to a large language model for processing.

The paper explores what it means to integrate AI agents into enterprise IGA programs. It highlights the potential role of the System for Cross-domain Identity Management protocol, which is currently the standard for automating user lifecycle management, syncing single sign-on systems with human resources platforms. As a user’s employment status changes, SCIM automatically updates their access rights across the organization’s IAM systems.

The same rigorous lifecycle management is equally critical for AI agents, requiring formal processes for their creation, permissioning, and eventual decommissioning. Experimental work is now underway to formally extend the SCIM protocol to support agent identities. By using an extended SCIM schema, organizations could provision agents into services just like human users. This enables centralized IT administration where agent permissions are governed by the same automated, policy-driven workflows used for employees, moving beyond risky ad-hoc processes.

The research discusses the various open standards that will be affected by elevating AI agents to first-class entities and the work required to retool these standards. This would provide IT managers with the improved visibility and control they desperately need over AI agent deployments within their organizations.

(Source: ZDNET)