OpenAI's Secrets 🤫: Danger & Billions 💰

AI

🎧

OpenAI’s Human Baseline Project Unveiled
OpenAI is undertaking a significant initiative to establish a human performance baseline for evaluating its next-generation AI models. The company recently enlisted third-party contractors to generate realistic tasks and assignments mirroring those performed in previous workplaces, as revealed by records obtained by WIRED and training data company Handshake AI. This project directly aligns with OpenAI’s broader efforts to measure AI performance against human professionals across diverse industries, signifying a crucial step in their pursuit of Artificial General Intelligence – an AI system capable of exceeding human performance at most economically valuable tasks. “We have hired individuals across various occupations to collect real-world tasks modeled after those you’ve completed in your full-time jobs, enabling us to accurately measure AI model performance on these tasks,” reads a confidential OpenAI document, highlighting the core strategy.

Realistic Tasks: A Contractor-Generated Data Set
The core of OpenAI’s approach involves contracting individuals to produce authentic, work-related documents and materials. Instead of simply summarizing their previous jobs, contractors are asked to provide concrete examples – such as Word documents, PDFs, PowerPoint presentations, Excel spreadsheets, images, or code repositories – demonstrating their skills and capabilities. This process is further supported by presentation notes illustrating realistic scenarios, and even fabricated work examples designed to test and demonstrate AI model responses. OpenAI and Handshake AI declined to offer direct comment on the initiative, further adding to the mystery surrounding the project’s scope and objectives.

Specific Example: The Luxury Yacht Trip Scenario
A concrete example of this process emerged during a presentation, showcasing a task assigned to a “Senior Lifestyle Manager at a luxury concierge company for ultra-high-net-worth individuals.” The task involved preparing a short, 2-page PDF draft of a 7-day yacht trip overview to the Bahamas for a family traveling there for the first time. The "experienced human deliverable" presented – a genuine Bahamas itinerary created for a client – demonstrated the contractor’s skills and provided valuable training data. Importantly, OpenAI explicitly instructs contractors to delete all corporate intellectual property and personally identifiable information from uploaded files, adding another layer of security and control.

Risk Mitigation: Legal Concerns and Data Scrubbing
Legal experts, such as Brown, an intellectual property lawyer with Neal & McDevitt, warn of potential trade secret misappropriation claims. AI labs receiving confidential information from contractors on a large scale face substantial legal risks. Contractors who provide documents—even those that have been scrubbed—from their previous workplaces risk violating nondisclosure agreements or exposing trade secrets. “The AI lab is putting a lot of trust in its contractors to decide what is and isn’t confidential,” Brown explains. “And if they do let something slip through, are the AI labs truly taking the time to determine what constitutes a trade secret? It seems to me that the AI lab is placing itself at significant risk.” The use of a ChatGPT tool, “Superstar Scrubbing,” highlights the effort being put into removing sensitive information before it’s submitted.

A Growing Industry and the Importance of Data Acquisition
AI labs are increasingly reliant on third-party contracting firms, such as Surge, Mercor, and Scale AI, to manage networks of data contractors, a trend driven by the need for higher-quality data to improve their models. This has spurred the growth of a lucrative sub-industry, estimated at $3.5 billion in 2022 for Handshake and reportedly valued at $25 billion during fundraising talks for Surge last summer. These firms—along with OpenAI, Anthropic, and Google—are hiring large numbers of contractors to generate training data for AI agents designed to automate enterprise tasks.

OpenAI's Data Sourcing Strategy: Direct Outreach
In exploring alternative methods for acquiring real company data, a consultant specializing in asset sales following business closures revealed to WIRED that an OpenAI representative had approached several firms regarding data acquisition. This individual, speaking on condition of anonymity to protect existing business relationships, stated that the OpenAI representative sought access to documents, emails, and other internal communications—provided that all personally identifiable information would be removed.

This article is AI-synthesized from public sources and may not reflect original reporting.