How a Global Insurance Company and Scale Are Building a Real-Time Copilot With Nemotron

Today we announced a collaboration with NVIDIA to help enterprises customize Nemotron open models through Scale Post-Training. This post takes a technical look at our post-training work with a global insurance company, and the real-time copilot we built for its customer service representatives.

When a driver calls the company for roadside assistance, the representative on the line has to understand what happened, work out which benefits the customer's policy covers and arrange the right help, all while the call is still going. The relevant facts come out over several turns, and the representative has to make important decisions before the call ends. Getting the answer right the first time builds customer trust and keeps costs in check.

We built a copilot that runs alongside the company's representatives on live French-language calls. As the conversation unfolds, it extracts the key facts of the case, recommends which benefits apply and flags any promise that would go beyond what the policy includes.

Post-training with supervised fine-tuning (SFT) and reinforcement learning (RL) made the model more accurate and cheaper to run:

• Fact extraction reached 93.8% accuracy, up from 76.1% before post-training and 31 points above the production model’s 62.6%.
• Coverage recommendation reached 95.6%, compared with 85.4% for the production model and 92.2% for the best frontier model we tested.
• Inference cost fell by roughly 70 to 75% compared with the production model, with under $1,000 in training compute per task.

The Workflow: Three Connected Tasks

We split the copilot into three tasks and evaluated each separately, so a gain in one couldn’t hide a failure in another.

• Extract case facts. From the conversation, the copilot identifies the event type, the breakdown reason and the number of passengers. It updates its answers as new information comes in, and leaves a field blank when the customer hasn’t yet said enough.
• Recommend coverage. Using those facts and the customer’s policy, the copilot identifies which benefit families may apply, such as roadside repair, towing, mobility or accommodation. If a deciding fact is missing, it asks for it instead of assuming the customer is covered.
• Guard against over-promising. As the representative speaks, the copilot checks whether a commitment exceeds the customer’s coverage. For example, it should flag a promise to reimburse a €60 taxi fare if the policy doesn’t cover it, even when the customer mentioned the amount several turns earlier.

Each task is difficult in its own way. Fact extraction requires reasoning over a conversation as it unfolds, while coverage recommendation depends on applying country-specific policy rules. The guardrail has to catch commitments in real time without interrupting the call with false alerts. Since representatives rely on this guidance while still on the line, the copilot also has to respond with low latency.

How We Evaluated

The first step in any post-training engagement is defining what “correct” means for the workflow. The company’s call data needs legal approval before it can be used for training, so we generated synthetic French conversations from sampled case facts and policy scenarios.

The scoring is strict:

• Fact extraction counts an answer as correct only when all three fields match exactly.
• Coverage recommendation counts an answer as correct only when the full set of recommended benefits matches. One extra or missing benefit makes it wrong.
• Guardrail counts a call as correct only when every check across the call matches the ground truth. Violations occur in only about 4% of checks, so the guardrail has to catch those rare cases without raising false alerts on the rest.

How We Post-Trained

For each task, we first applied SFT on the model’s own correct reasoning and tool-use trajectories, selected from multiple attempts. We then applied RL, using group sequence policy optimization (GSPO), starting from the SFT checkpoint, and rewarded exact matches with the verified answer. SFT widened the range of cases the model could solve. RL made it more consistent on the first try.

Training was inexpensive. The fact-extraction model took about 16 GPU hours on 8 H100s (about $890), and the coverage model took about 4 GPU hours (about $230).

Results

Fact extraction. Base Nemotron started at 76.1%, already ahead of the production model’s 62.6%. It understood the French conversations well; its main difficulty was applying the rules that connect an event to a breakdown reason. After SFT and RL, it reached 93.8%, in line with the strongest frontier models we tested (95.8% and 94.4%).

Coverage recommendation. Base Nemotron scored 93.6%, the highest of any model before training. After post-training it reached 95.6%, ahead of every model we tested.

Cost and Latency

Because the copilot runs during a live call, accuracy only matters if the model is also fast and affordable at scale. Nemotron 3.5 Super VL achieves frontier capability at lower latency by using an MoE architecture that only activates 12B parameters out of 120B total:

• Fact extraction: about $3.00 per 1,000 queries, compared with $12.10 for the production model, roughly 75% lower. At a lower reasoning setting, it still scores 91.9% at $1.80 per 1,000 queries, and latency drops from 26 seconds to 11.
• Coverage recommendation: about $1.40 per 1,000 queries, compared with $4.80 for the production model, roughly 70% lower.

What We Learned

Post-training doesn’t guarantee improvement. The right recipe depends on the task, so we measure every change against the full workflow.

Inference settings matter as much as training. A cheaper, faster configuration gave up less than two points of accuracy on fact extraction.

Evaluation makes cost savings possible. A task-level evaluation answers a practical question: what is the least expensive setup that meets this workflow’s quality bar?

Along the way, our engineers contributed improvements back to NVIDIA’s open post-training tools to support reinforcement learning on Nemotron’s hybrid mixture-of-experts architecture.

What’s Next

These results come from synthetic calls in a controlled setting. Next, we’ll evaluate on approved real calls, measure latency in the live workflow and finish the guardrail work before bringing the copilot into production.

This engagement is one of several early deployments of Scale Post-Training, alongside our work with Skydance on finance automation and EY on M&A due diligence.

Scale Post-Training is available now for enterprises that want to customize open models for their own workflows, including the Nemotron model families. To learn how we can help your enterprise, contact us here.