Microsoft open-sources new tools for building and testing AI agents
▼ Summary
– Microsoft’s AI Red Team released two open-source tools: Clarity for structured design review and RAMPART for continuous security testing of AI agents.
– RAMPART is a test harness for CI pipelines that runs automated adversarial tests (e.g., prompt injection) and returns pass/fail signals, with support for multiple runs and pass thresholds.
– Clarity guides engineers through structured design conversations covering problem clarification, solution exploration, failure analysis, and decision tracking, outputting markdown files.
– RAMPART was used by Microsoft’s AI incident response team to generate 100 variants of a vulnerability, test mitigations, and reduce remediation time from weeks to hours.
– Both tools have been battle-tested internally at Microsoft and are now available on GitHub, continuing Microsoft’s practice of open-sourcing AI security tools like Counterfit and PyRIT.
Microsoft has opened up two internal security tools designed to bring rigorous safety discipline to AI agent development. The company’s AI Red Team, which stress-tests its own systems, released Clarity, a structured design review tool, and RAMPART, a continuous testing framework. Both have been battle-tested internally before being shared publicly.
RAMPART functions as a test harness built on top of PyRIT, Microsoft’s existing open-source red-teaming library. It plugs directly into the CI pipeline, allowing developers to write pytest tests that simulate adversarial scenarios, such as prompt injection attacks. These tests run automatically on every code change.
“Each test connects to the agent through a thin adapter, orchestrates an interaction, and evaluates observable outcomes. Tests return a clear pass or fail signal and can be gated in CI just like any other integration test. When a new tool or data source is added to the agent, the corresponding safety test can be added in the same pull request,” said Ram Shankar Siva Kumar, founder of Microsoft’s AI Red Team.
Because AI systems are probabilistic, RAMPART supports running the same test multiple times and setting a pass threshold. Kumar noted that Microsoft’s AI incident response team has already used the tool to address real-world vulnerabilities. “They took a reported vulnerability, generated 100 variants of the vulnerability via RAMPART and used it to test the potency of each of these variants,” he explained. “Engineers then applied mitigations to all variants, and tested if the mitigations work via RAMPART. What we found is that work that would have taken Microsoft experts weeks can now be done in hours with RAMPART.”
Clarity addresses a different challenge: catching costly design decisions early. It guides engineers through structured conversations covering problem clarification, solution exploration, failure analysis, and decision tracking. “It asks the kinds of questions that experienced architects, product managers, and safety engineers would ask, the ones that are easy to skip when a team is excited about building something new,” Kumar said.
The output is saved as human-readable markdown files in a `.clarity-protocol/` directory, making it easy for teams to review later. The failure analysis component is especially thorough, using multiple AI “thinkers” to examine the system from different angles, including security, human factors, adversarial scenarios, and operational concerns. Teams then work through results together, grouping related failures, tracing causal chains, and building management plans.
Microsoft has a history of open-sourcing its internal AI security tools, including Counterfit in 2021 and PyRIT in 2024. PyRIT now has over a hundred external contributors and is regularly updated by both Microsoft and the community. “We are releasing RAMPART and Clarity because these have been battle tested with Microsoft engineers,” Kumar told Help Net Security. “Given the speed of AI world, we are sharing these tools with the community today because we want these early projects to benefit everyone.”
Both RAMPART and Clarity are now available on GitHub.
(Source: Help Net Security)