Scale AI Partners with Singapore's IMDA to Advance AI Evaluation for Southeast Asia

As AI systems grow more capable and more widely deployed, the ability to test, evaluate, and verify them within national contexts has become foundational.

Because of this, in April of this year Scale AI and Singapore’s Infocomm Media Development Authority (IMDA) formalized a collaboration framework to advance AI evaluation research, with an initial focus on building a large-scale legal benchmark for Singapore law.

Global valuations should account for national and regional contexts, especially in regions that remain underrepresented in today’s benchmarks.

Today, we are excited to share that our first joint research initiative with Singapore’s IMDA is the development of one of the first large-scale legal benchmarks for Singapore law. This benchmark will measure not only whether models understand Singapore law, but whether they can identify when legally relevant facts are missing, ask useful clarification questions, and respond usefully to realistic legal scenarios grounded in Singapore law.

Beyond the legal benchmark, Scale AI and IMDA are exploring opportunities for technical research and information sharing, as well as policy-relevant research, across topics such as agentic AI, model evaluation methods, tools, benchmarks, and multilingual AI safety.

A Shared Commitment to Contextual Evaluation

Scale’s work with frontier model developers, governments, and enterprises has surfaced a consistent challenge: AI systems often fail to perform reliably outside the environments in which they were most heavily trained and tested — behaving unpredictably and exposing significant capability gaps.

IMDA brings deep expertise in digital governance and AI policy, experience in AI verification, and an on-the-ground understanding of AI deployment across Singapore and Southeast Asia's diverse societies. Southeast Asia is one of the world’s most linguistically diverse and digitally connected regions, yet remains underrepresented in many frontier AI benchmarks. That gap creates meaningful blind spots in how model safety and performance are measured. As Singapore's lead agency for the development and regulation of the infocomm and media sectors, IMDA has been at the forefront of responsible AI, including through initiatives like AI Verify and Singapore's Model AI Governance Framework(s).

Areas of Collaboration

The collaboration will focus on two areas:

• Joint evaluation research: Developing benchmarks and evaluation pipelines, with an initial focus on Singapore law and potential lessons for Southeast Asia's legal, linguistic, and cultural contexts, ensuring that evaluation infrastructure reflects the full diversity of AI deployment environments, not just the most studied ones.
• Open research: Publicly releasing a subset of these benchmarks and evaluations, where possible, to support future research and enable independent replication.

The Future of AI Evaluation is Global

The most capable AI systems are trained and deployed across borders. But evaluation gaps often emerge along those same borders, in jurisdictions, languages, and legal systems that remain underrepresented in training data and safety research.

We are proud to work with IMDA on this effort. Scale AI is committed to building an evaluation ecosystem that is rigorous, contextually grounded, and globally relevant, and to expanding collaborations like this one as a core part of that mission.