Major Vendors Quit MITRE Security Evaluations
▼ Summary
– Microsoft, SentinelOne, and Palo Alto Networks will not participate in the 2025 MITRE ATT&CK Evaluations, raising concerns about the test’s future relevance.
– The companies cited a desire to prioritize product development and innovation as the official reason for withdrawing from the test.
– MITRE’s CTO suggested the test may have become too difficult and resource-intensive for vendors, while experts believe it is now seen as promotional rather than driving real security gains.
– The 2025 evaluation will feature complex scenarios, including cloud environment testing, which requires significant vendor attention and resources.
– MITRE plans to re-establish a vendor forum for the 2026 test to improve collaboration and set objectives with the industry.
The cybersecurity landscape is facing a significant shift as three major vendors have withdrawn from the 2025 MITRE Engenuity ATT&CK Evaluations. Microsoft initiated the exodus in June, followed by confirmations from SentinelOne and Palo Alto Networks in September. This collective departure has sparked intense discussion within the industry regarding the future relevance and direction of these highly regarded endpoint detection and response (EDR) tests.
For Microsoft, the decision is particularly noteworthy. The company had previously leveraged its strong performance in these evaluations to actively market its Microsoft Defender XDR platform. All three organizations have publicly attributed their withdrawal to a strategic need to reallocate resources toward product development and innovation. However, industry observers speculate that other motivations may be influencing these moves, including a growing perception that the tests have become more about marketing than about driving substantive security improvements.
The MITRE ATT&CK framework, introduced in 2015, rapidly became an industry standard for categorizing the techniques, tactics, and procedures (TTPs) used by cyber adversaries. The Evaluations program launched in 2019 with the goal of bringing more rigor and consistency to third-party security testing. According to Charles Clancy, MITRE’s Chief Technology Officer, the program was designed to address a market gap where inconsistent methodologies failed to push the industry forward effectively. The annual Enterprise evaluation is considered a premier event, sometimes described as the “Olympics of cybersecurity.”
The testing process involves MITRE’s team selecting real-world threat actors and using their own Caldera platform to simulate attacks against participating vendors’ EDR solutions. Performance is measured on criteria like detection accuracy and false positive rates. Clancy emphasizes that the test is not intended as a longitudinal benchmark, as each year features a completely different adversary simulation to reflect the evolving threat landscape.
The 2024 evaluation tested vendors against threats from North Korean-affiliated hackers and the CL0P and LockBit ransomware groups. That year saw CrowdStrike absent, with speculation linking its non-participation to a major global outage it experienced shortly before the test. The upcoming 2025 evaluation will feature two complex scenarios: a financially motivated attack on a hybrid environment and a Chinese-aligned cyber-espionage campaign.
Despite these plans, the absence of Microsoft, SentinelOne, and Palo Alto Networks is a major development. Their official statements uniformly cite a need to focus engineering efforts on customer-facing innovation. Clancy suggests he understands the deeper reasons, pointing to the significant resource commitment required and the fact that his team intentionally makes the test more challenging each year. He conceded that the balance may have been pushed too far for 2025, making participation particularly demanding.
Further insights come from security professionals. A senior product manager at ManageEngine noted that recent changes, like the 2024 introduction of alert volume metrics, increased the difficulty for vendors by highlighting alert fatigue. The addition of a cloud environment scenario for 2025 represents another layer of complexity in untested territory. Clancy also acknowledged that a key vendor forum, which helped align test objectives with the industry, had lapsed in recent years, potentially contributing to a disconnect.
Critical voices on platforms like LinkedIn argue that the evaluations have devolved into “vendor theater,” where the primary goal for participants has become public relations victories rather than genuine security enhancements. They suggest that with MITRE and CISA facing budgetary pressures, some vendors saw an opportunity to step back from what they perceive as an outdated, overly endpoint-focused exercise.
In response to the criticism and withdrawals, Clancy has announced plans to re-establish the vendor forum for the 2026 evaluation cycle. This move is seen as an effort to rebuild collaboration and ensure the test remains valuable and aligned with real-world security needs. Despite the high-profile departures, a dozen other cybersecurity vendors are confirmed to be participating in the 2025 tests.
(Source: Info Security)