We found the standard approach to toolchaining insufficient. Simply giving the LLM access to multiple tools and asking it to clean the data led to extremely poor results, if it could do it at all. Instead, when we gave the LLM access to a python sandbox pre-loaded with these tools and asked it to develop the aforementioned plan, the output was significantly improved. For the remainder of this blog, we dive into why this happened, how we set up our experiment, and what the findings mean for you.

How do you know if an AI model is actually learning, or just getting better at faking it? A new paper from researchers at Scale introduces Rubrics as Rewards (RaR), a framework that solves this problem by training models with structured, expert-designed checklists instead of simple preference scores. This approach moves the human role from a simple preference labeler to an expert architect of the AI's values, resulting in up to a 28% performance leap on challenging benchmarks and providing a more transparent, effective path toward reliable AI.

Scale AI has been awarded a $99 million contract by the U.S. Department of Defense to accelerate Army research and development in artificial intelligence. Building on its expanding partnership with the Pentagon, Scale will deliver data operations, platforms, and engineering support to help the Army adopt AI across critical missions.

In today’s episode Scale’s enterprise team (Clemens Viernickel, Mark Pfeiffer, Sam Denton, and Felix Su) review several viral AI agent demos from the internet and decide how realistic they are in their current form to be deployed in an enterprise environment. What do you think of their votes?

AI is moving beyond text, toward agents that can listen, speak, and interact naturally with the world. Voice AI requires far more than word; it demands the nuanced tones, emotions, and dynamics of human speech. But unlike text, there’s no vast public library of labeled audio to train on. Scale is building that foundation, delivering high-quality, diverse, and emotionally rich speech data to power every stage of model development. From real-time conversation to multimodal perception, these datasets are unlocking the next era of human-computer interaction. The future is listening.

Agentic AI is the most exciting breakthrough Scale is pioneering.  As a world leader in Agentic AI, Scale is putting AI agents to work that operate autonomously and are tireless in their ability to perform complex tasks.

Language models provide impressive general capabilities off the shelf, but because they were not trained on private enterprise data they fall short of delivering the specialized performance enterprises need for their unique workflows, internal systems, and proprietary data. At Scale, our Safety, Evaluations and Alignment Lab (SEAL) recently published foundational reinforcement learning research, and our enterprise team is pioneering new approaches to solve this challenge through cutting-edge reinforcement learning research focused on training AI agents specifically for enterprise environments.

A fundamental shift is underway in how AI for healthcare is evaluated. Recent studies from OpenAI, Google, and Microsoft move beyond simple accuracy scores to establish a new standard for measuring an AI's healthcare skills. This post provides an analysis of three distinct evaluation methodologies that redefine what "good" looks like for clinical AI. We explore how OpenAI's HealthBench uses a massive, rubric-based system to measure foundational safety; how Google's AMIE tests the nuanced, "soft skills" of an interactive diagnostic dialogue; and how Microsoft's SDBench validates an agent's ability to make strategic, cost-conscious decisions. By examining these benchmarks and their results, we provide a glance at the future of AI in healthcare.

The shift from reactive models to agentic systems fundamentally alters the AI risk landscape, making frameworks that focus only on user intent and model output incomplete. To address this gap, we've evolved the AI Risk Matrix by adding a crucial third dimension: Model Agency. This article breaks down agency into three tiers—Tools, Agents, and Collectives—using concrete examples to illustrate how complex failures can now emerge from the system itself. We argue that this systemic view, which connects model behavior to traditional AppSec vulnerabilities, is essential for building the next generation of safe and reliable AI.

We peer into our crystal ball and try to predict the next 6 months in enterprise AI. Are you surprised by any of these predictions?