OpenAI Asks Contractors to Share Past Work for AI Agent Testing
▼ Summary
– OpenAI is collecting real workplace tasks and deliverables from contractors to establish a human performance baseline for evaluating its next-generation AI models.
– The company’s goal is to measure AI performance against human professionals across industries, which it views as a key step toward achieving Artificial General Intelligence (AGI).
– Contractors are instructed to upload actual work files (like documents or presentations) from current or past jobs, though they can also submit fabricated examples based on realistic scenarios.
– OpenAI directs contractors to remove confidential information, proprietary data, and personal details from uploaded files, mentioning a tool called “Superstar Scrubbing” for this purpose.
– Legal experts warn this practice risks trade secret misappropriation and could violate non-disclosure agreements, potentially exposing both contractors and OpenAI to legal liability.
OpenAI is actively seeking real-world work examples from third-party contractors to benchmark its advanced AI models against human performance. This initiative, part of a broader evaluation framework, aims to establish a human baseline for complex professional tasks across various industries. The company views this comparative analysis as a critical step in its progress toward developing artificial general intelligence (AGI), defined as a system capable of outperforming humans in most economically valuable work.
According to internal documents, OpenAI has instructed contractors to upload detailed examples of assignments they have completed in their current or past roles. These submissions are intended to be authentic pieces of work that required hours or days of effort, transformed into specific tasks for AI evaluation. The company requests the actual files produced, such as Word documents, PDFs, presentations, or spreadsheets, rather than summaries. In some cases, contractors may also submit fabricated examples designed to realistically simulate how they would handle a given professional scenario.
The project focuses on capturing both components of a real-world task: the initial request from a manager or colleague and the final deliverable created in response. OpenAI’s guidelines repeatedly stress that shared examples must reflect genuine, on-the-job work the contractor has personally executed. One illustrative example involves a senior lifestyle manager at a luxury concierge service preparing a detailed seven-day yacht itinerary for a family’s first trip to the Bahamas.
To address confidentiality concerns, OpenAI directs contractors to remove or anonymize all sensitive information before submission. This includes personal data, proprietary business information, internal strategies, and any material nonpublic details. The company has even referenced an internal tool designed to advise on scrubbing confidential content from documents.
However, legal experts highlight significant risks in this data collection approach. By relying on contractors to determine what constitutes confidential information, AI companies may expose themselves to potential trade secret misappropriation claims. Contractors who share documents from former employers, even after redaction, could violate non-disclosure agreements or inadvertently disclose protected secrets. The scale of this operation places considerable trust in individual contractors’ judgments, with AI labs potentially facing substantial legal liability if sensitive information slips through the vetting process.
(Source: Wired)
