Earlier this year, I made the exciting decision to join Scale. This move marks the next chapter in a journey I've been deeply passionate about for years—exploring how we can harness the power of AI to create meaningful opportunities for people around the world.

Training AI models to behave responsibly in the real world means preparing them for the full range of online content — including the challenging parts. It’s not easy work, but it’s necessary. At Scale, we believe that building AI systems that avoid harmful, abusive, or dangerous behavior is one of the most important challenges of our time. And we’re proud to support the people who make this possible.

Welcome to Human in the Loop with Scale AI. We're kicking off with an episode diving into the current AI agent landscape and covering what’s important for enterprises to move beyond demos to real, reliable agentic systems.

As part of Scale’s ongoing investment in its AI workforce in St. Louis, Scale and the University of Missouri-St. Louis (UMSL) are officially launching a collaborative education effort.

When we evaluated o3 and o4-mini on Humanity’s Last Exam, we noticed their calibration errors were significantly lower than predecessors. A well-calibrated model is like someone who knows when they are likely to be right or wrong. If a well-calibrated model says it’s 70% confident on a set of questions, it should be correct about 70% of them. Calibration error measures this difference between the model’s stated confidence and its actual accuracy – ideally it’s 0%. All models benchmarked so far have exhibited much higher calibration errors. Are the newer generation of reasoning models from OpenAI truly better calibrated?

As LLMs become more sophisticated, maintaining a distinct human voice isn't just stylistic—it's essential. Explore why your unique perspective matters more than ever and learn actionable techniques for working with LLMs to enhance your writing process while keeping your authentic voice front and center.

Since its inception in 2023, Outlier has become a cornerstone of the AI industry—connecting hundreds of thousands of people across the globe with meaningful and flexible work. Hailing from cities and small towns across the world, Outlier contributors have earned a combined hundreds of millions of dollars to help build the foundation of today’s most advanced AI models

Frontier AI development has reached an inflection point: as models rapidly advance in capabilities, the need for sophisticated evaluation has become a decisive factor in competitive success. That’s why today we're announcing updates to Scale Evaluation, our platform that helps teams identify model weaknesses and validate improvements. Our updated platform introduces four key capabilities: instant model comparison across thousands of tests, multi-dimensional performance visualization, automated error discovery, and targeted improvement guidance—all designed to help teams identify weaknesses faster and make more confident release decisions. These updates build on Scale Evaluation’s foundation introduced last year, broadening access to frontier evaluation capabilities.

Scale AI products have been approved for purchase on AWS Marketplace for the U.S. Intelligence Community (ICMP). ICMP is a digital catalog that makes it easy for customers in the U.S. national security community to find, test, buy, and deploy software that runs on AWS.

The next four years will be critical to the future of AI leadership around the world and the case for bold action has never been clearer. More than one year ago the United States was leading the world in the development of AI systems, but today that is no longer the case. Chinese AI advancements, most notably with the launch of Deepsek, have shown that China has closed the gap and now the race is nearly tied. It is not enough for the United States to match China’s intensity on AI, we must exceed it or simply put, we lose. President Trump rightly called Deepseek’s release “a wake up call” and now, the US needs to heed the call to action and determine how to best respond in order to win.