As the capabilities of AI systems rapidly increase, it is clear that AI holds a great deal of promise for transforming our world for the better. At the same time, similar to many scientific advancements before it, AI also harbors the potential for malicious use. That is why in 2023, Scale published our&nbsp;<a href="https://scale.com/guides/test-and-evaluation-vision">vision for model test &amp; evaluation</a>, followed by our new frontier research effort, the <a href="https://scale.com/blog/safety-evaluations-analysis-lab">Safety, Evaluations and Alignment Lab</a> (SEAL).
Recently, <a href="https://www.whitehouse.gov/briefing-room/presidential-actions/2023/10/30/executive-order-on-the-safe-secure-and-trustworthy-development-and-use-of-artificial-intelligence/">the White House Executive Order on Artificial Intelligence</a> highlighted the risks of LLMs in facilitating the development of bioweapons, chemical weapons, and cyberweapons. Unfortunately, evaluation of such hazardous capabilities has been limited, manual, and only possible for those with relevant domain expertise (or, those with sufficient resources to acquire this expertise).
That&rsquo;s why, today, in partnership with the <a href="https://www.safe.ai/">Center for AI Safety</a>&mdash;the creators of the industry-standard Massive Multitask Language Understanding (<a href="https://arxiv.org/abs/2009.03300">MMLU</a>) benchmark&mdash;Scale is publishing a novel safety evaluation benchmark for large language models: <a href="https://arxiv.org/abs/2403.03218">the Weapons of Mass Destruction Proxy (WMDP)</a>. Covering knowledge across biosecurity, chemical security, and cybersecurity, WMDP serves as an open source proxy measurement for hazardous knowledge contained by LLMs within these domains.
In developing WMDP, a top priority was ensuring that this research would not unintentionally publish hazardous information. To that end, none of WMDP&rsquo;s 4,157 questions are direct info hazards &ndash; the questions were developed in collaboration with a consortium of academics and technical consultants to focus on what we call precursor, correlated, or component knowledge &ndash; in other words, these questions delve into foundational and associated knowledge that is a step away from sensitive or risky information, without actually crossing into hazardous territory (see Figure 1).
<img src="https://lh7-us.googleusercontent.com/gZ0tOTg1xi93A37WIhA6pgX9lPxcoa7WSWb9hOb2VFQMuVYIk1KAxx9jNSboj-Wftd6LkPX132E1N93JXf21LxBkPougYvNOgq49mC-yI3CzI6ToGz2Ol6CwQgV8DLP_Wk1oH5qFTbWnPv2qBmiy2zI" width="624" height="225">
Figure 1: In the left panel, research that aims to develop enhanced potential pandemic pathogens (ePPPs) is a precursor to developing novel viruses. In the center panel, topics in chemistry (e.g., procurement or synthesis) contain questions with a wide variance in hazard level, so we approximate especially sensitive information by collecting questions near the boundary. In the right panel, a cyberweapon requires knowledge of several components, so testing for knowledge of components (esp. those that are primarily offensive in nature) can approximate hazardous knowledge.
&nbsp;
In interpreting model results against WMDP, it is important to keep in mind what the benchmark measures and what it does not. It measures correlated, precursor, or component knowledge to hazardous topics, meaning that if models lack the knowledge covered by WMDP, then they likely lack a substantial amount of hazardous knowledge across the relevant domains. If models do demonstrate knowledge in WMDP, then the likelihood that they contain hazardous knowledge is higher. However, even models with a substantial amount of hazardous knowledge may still lack other requisite capabilities to combine that knowledge in the sequence of steps needed to present a danger.
While measuring risks in LLMs is critical, it&rsquo;s only the first step towards creating safer, more trusted models. It is also extremely important that the broader AI community continues to advance the state of research on how we might act to mitigate these risks and reduce the chances that a model could be leveraged for misuse by a bad actor. To that end, in collaboration with CAIS, we have leveraged WMDP as a benchmark for machine unlearning methods in removing hazardous knowledge from LLMs. We expect methods that lead to unlearning on WDMP to also unlearn the actual hazardous knowledge in these domains.&nbsp;
We hope that WMDP will serve as a foundational step in enabling further study of the problem of mitigating hazardous information in LLMs, and help to defend against LLM misuse while maintaining models&rsquo; educational and productive capabilities.
To read more about the WMDP benchmark and the work Scale and the Center for AI Safety are introducing today, you can reference the full paper&nbsp;<a href="https://arxiv.org/abs/2403.03218">here</a>.

The Weapons of Mass Destruction Proxy (WMDP) is an open-source measure of hazardous knowledge in large language models.