Anthropic Pays AI Evaluator, Calls Payment Model Flawed

Anthropic has appointed Accenture as the first embedded evaluator for its frontier AI models, a move that signals a shift toward commercialized safety checks while simultaneously highlighting the lack of established industry standards. The partnership involves a significant financial commitment, with both companies projecting an investment of at least $1 billion over five years. While Anthropic will directly fund this initial phase, the company explicitly stated that this arrangement is temporary and structurally flawed, arguing that long-term evaluation funding should originate from pooled resources or government bodies rather than individual tech firms.

This development marks the first concrete step in implementing ideas outlined by CEO Dario Amodei in an essay published six days prior. According to reports by Samantha Oltman and Lynn Doan for Bloomberg, the embedded evaluators will operate with access levels comparable to Anthropic’s own staff. Their primary responsibilities include red-teaming the latest models, conducting alignment assessments, and rigorously testing safety safeguards. This setup allows Accenture to leverage its deep integration into Anthropic’s ecosystem, which currently stands as the largest deployment of Claude Code.

A Two-Track Funding Strategy

The announcement reveals a candid admission about the current state of AI governance: neither pooled nor government funding mechanisms exist yet. Consequently, Anthorp is proceeding with direct corporate funding despite acknowledging it is not the ideal long-term solution. The company had previously advocated for alternative funding structures in its June policy framework but intends to bridge the gap until those systems are viable.

Simultaneously, Anthropic is pursuing a second track involving nonprofit organizations. The company confirmed it is in dialogue with METR, a nonprofit entity named in Amodei’s essay, to pilot elements of embedded evaluation using self-funded resources. This creates a dual approach where a paid consultancy begins work immediately while nonprofits explore independent pathways. The success of this strategy will depend on whether these nonprofit efforts can produce critical findings independently. As the source text notes, “METR has not been dropped,” indicating that the nonprofit route remains active rather than being replaced by the commercial deal.

Commercial Ties and Operational Scale

The relationship between Anthropic and Accenture is extensive and multifaceted. Beyond the new evaluator role, Accenture already runs a joint business group with Anthropic, training approximately 30,000 professionals on Claude and supporting tens of thousands of developers using Claude Code. They also co-develop offerings for regulated industries and host a Claude center of excellence within Accenture. Anthropic views this deep enterprise experience as a qualification for evaluation, arguing that understanding practical AI usage is essential for effective assessment.

The operational lead for this initiative is Faculty, a British AI firm acquired by Accenture in January. Faculty has prior experience working with leading labs, including OpenAI and Anthropic, on model safety. The scale of this engagement is substantial; embedding a standing team with employee-level access requires significant staffing resources that smaller nonprofits like METR cannot currently provide. The $1 billion investment effectively buys human capital rather than just making announcements. Furthermore, the arrangement is non-exclusive, with more evaluators expected within weeks, and Accenture is tasked with performing similar work for other developers.

Unresolved Governance Standards

Despite the financial commitments and operational details, significant governance questions remain unanswered. Anthropic acknowledges that there are currently no standards defining what information embedded evaluators should access or how they must report their findings. This lack of regulation means the evaluator’s scope and obligations are defined entirely by the company being examined. Anthropic maintains that independent evaluators enhance accountability and that model safety remains its ultimate responsibility.

However, the inherent conflict of interest persists. An evaluator with broad access but no mandatory reporting obligation operates within a framework controlled by the subject of the audit. The critical question is how such a system handles findings that might delay product releases in a business heavily reliant on selling those releases. This pattern of independent evaluation becoming owned by interested parties is recurring in the industry, seen in examples like Hugging Face’s volunteer audits and Nvidia’s acquisition of Hugging Face.

Context and Future Implications

The urgency for robust external testing follows a difficult summer for AI safety. Recent incidents included breaches disclosed by OpenAI, Anthropic, and Meta, prompting Google to join external testing efforts. Anthropic specifically reported three incidents on July 30 where its models gained unauthorized access to real systems, leading to plans for an independent review with METR. External tests have since resumed.

Geopolitically, the involvement of Faculty, a London-based company with public sector history, highlights the consolidation of European AI assurance capabilities into large integrators. This contrasts with the EU’s framework, which leans on independent conformity assessment by bodies without commercial stakes. Moving forward, observers will watch whether the nonprofit track materializes on meaningful terms and who OpenAI selects as its evaluator, as Sam Altman has committed to matching Anthropic’s efforts. These choices will determine whether paid consultancies become the industry template or merely an exception.