Building an AI agent that works is hard. Making it work better is harder, and it's slow, manual engineering work. Scale researchers built the VeRO (Versioning, Rewards, and Observations) framework to test whether AI can take some of that work on. With VeRO an "optimizer" agent can edit any part of a target agent. It improves areas where success depends on tool use and workflow logic, but not areas where success depends on the underlying model's reasoning ability.
Researchers from Scale Labs will present VeRO at the International Conference on Machine Learning in Seoul on July 7.
What Worked and What Didn’t
Across 105 optimization runs, individual gains reached as high as 19 points on a benchmark measuring multi-step tasks with heavy tool use (GAIA). The pattern behind those wins: optimizer agents are good at improving how a target agent interacts with the world, and bad at improving how it thinks.
We tested three optimizer agents, powered by Claude Sonnet 4.5, Claude Opus 4.5, and GPT-5.2-Codex, with the target agents built on GPT-4.1 mini. Tool use, file handling, search, and workflow logic all got meaningfully better. When the limit is the underlying model's reasoning ability, no amount of tweaking around it helps.
The improvements are real. The harder question is whether they hold up outside the test conditions.
How the Wins Hold Up
A higher score doesn't always mean a better agent. Scores can bounce around from run to run, so a gain might just be luck. The optimizer might have gotten good at the specific test questions it practiced on without learning anything that works on new ones. Or it might have found a shortcut in how the test is scored that doesn't reflect any real improvement.
VeRO is built to catch these problems. Every change to the target agent is saved as a separate version, so it's clear which edits produced the gains, and unsuccessful ones can be undone. The optimizer is capped at a fixed number of attempts, so it can't win by sheer trial and error. And it never sees the questions used for the final score, so it can't tune itself to them.
With VeRO’s guardrails in place, agent improvement can be verified.
Which Wins Carry Over
The hardest test of an optimized agent is whether it still works after the model underneath it gets upgraded. In a real deployment, models change, APIs change, tools get added. A fix that only works in the exact setup where it was discovered is a liability.
So the best-performing agents were rerun against different models, including different sizes, families, and vendors. The pattern was consistent: structural changes (new tools, modified workflows) held up. Prompt edits were uneven, sometimes performing worse than the unoptimized baseline.
That matters because optimizers reach first for the prompt. More than half of all changes were prompt edits, even when tools and workflows were on the table. They're easy to make and the least likely to survive a model swap; the gains worth keeping are structural.
Engineering Work is Changing
The shape of agent engineering work is changing. The parts that respond to trial and error are increasingly things AI can handle. The parts that depend on judgment, on reasoning about what an agent should do in the first place, still don't. Knowing in advance which changes will survive the next model upgrade is still an open problem, and building the discipline to tell durable gains from illusory ones is the work that comes next.
For the technical detail behind the framework, see the paper. For the engineering walkthrough and the full set of findings, see the Scale Labs blog.
Ready to break through your data bottleneck?
Scale's team will match your project to the right experts, fast.
