AI Safety is Broken? How Code is Fixing the Internet 💻✨
AI
🎧
After leaving Apple in 2019, Brett Levenson observed that content moderation was unsustainable, noting human reviewers had only about 30 seconds per piece of flagged content. This challenge intensified with AI chatbots, which led to safety issues like self-harm guidance. This prompted the concept of "policy as code," leading to the founding of Moonbounce. The company, which announced raising $12 million on Friday, provides an additional safety layer utilizing a trained large language model to evaluate content in 300 milliseconds or less. Moonbounce supports multiple verticals, and its next focus is "iterative steering," a capability designed to modify prompts in real time to redirect chatbots toward supportive responses.
THE FAILURES OF TRADITIONAL CONTENT MODERATION
When Brett Levenson left Apple for Facebook in 2019, he found that content moderation problems were far deeper than simple technological fixes. The old system required human reviewers to memorize complex, 40-page policies that had been machine-translated. Furthermore, these reviewers were given only about 30 seconds per flagged piece of content to make critical decisions—whether to block, ban, or limit—with an accuracy rate only slightly better than a coin flip. This reactive, delayed approach was inherently unsustainable, a problem exacerbated by the rise of nimble, well-funded adversarial actors and sophisticated AI chatbots.
THE NEED FOR POLICY AS CODE
The increasing complexity and failure points in content moderation became critical issues, highlighted by high-profile incidents such as chatbots giving self-harm guidance or AI-generated imagery successfully evading safety filters. Recognizing the limitations of manual review, Levenson developed the concept of "policy as code," a revolutionary approach that translates static policy documents into executable, updatable logic tightly coupled directly to the enforcement mechanisms. This insight formed the foundation for the founding of Moonbounce, a company that subsequently announced a $12 million funding round, co-led by Amplify Partners and StepStone Group.
MOONBOUNCE’S REAL-TIME ENFORCEMENT ENGINE
Moonbounce functions by providing an advanced safety layer wherever content is generated, whether the source is a standard user or an advanced AI model. The company has trained its own large language model to analyze a customer’s policy documents and evaluate incoming content at runtime. This system delivers a decision in 300 milliseconds or less, taking immediate action that can range from slowing down content distribution while awaiting human review, to instantly blocking high-risk material.
SCALING SAFETY ACROSS MULTIPLE VERTICALS
The platform supports three primary industry verticals: large platforms utilizing user-generated content, such as dating apps; AI companies developing characters or companions; and advanced AI image generators. Moonbounce is currently supporting over 100 million daily active users and managing more than 40 million daily reviews. Key customers leveraging this technology include the AI companion startup Channel AI, the image/video company Civitai, and roleplay platforms like Dippy AI and Moescape.
THE ADVANTAGE OF THIRD-PARTY GUARDRAILS
Industry experts like Tinder’s head of trust and safety have noted that while LLMs present unprecedented opportunities, they also create daunting moderation challenges. Moonbounce provides a crucial third-party advantage: unlike the AI chatbot itself, which must process and remember potentially tens of thousands of tokens of preceding context, Moonbounce's system is solely focused on objective, real-time enforcement of rules. This capability allows companies to transform safety from a mere afterthought into a core product benefit and competitive differentiator.
NEXT-GENERATION AI SAFETY CAPABILITIES
The company’s future development focuses on "iterative steering," a capability designed to handle sensitive interactions that require more than a simple refusal. In response to tragic incidents, Moonbounce aims to intercept conversations and redirect them in real time, modifying the user's original prompt to guide the chatbot toward a more actively supportive and helpful response, rather than simply refusing to engage with the topic.
COMPANY VISION AND STRATEGIC FOCUS
Levenson runs the 12-person company alongside his former Apple colleague, Ash Bhardwaj, who built large-scale cloud and AI infrastructure for the iPhone-maker. While acknowledging that Moonbounce would fit perfectly within the stack of his former employer, Levenson emphasized his fiduciary duty and deep commitment to the technology, stating his strong reluctance to see the system acquired and then restricted by a single entity.
This article is AI-synthesized from public sources and may not reflect original reporting.