icon Scale Is Excited To Be OpenAI’s Preferred Partner for Fine-Tuning GPT-3.5!Learn More →

Goodcall Boosts Call Quality with Scale Rapid

Overview

Goodcall Leverages Scale Rapid to Improve their State-of-the-art Chatbots

Goodcall provides businesses with intelligent phone agents so that their customers can always get in touch. When customers call these bots to request information like the business’s hours, location, and products or services, these phone agents perform automatic speech recognition (ASR) to convert speech-to-text. Then, once the text is acquired, the bots employ AI analysis to interpret the person’s request and to provide the appropriate response. The intent classification models that power Goodcall’s chatbots need to be regularly fine-tuned with real-world production data. Any performance improvement in the model also means an improvement in the customer experience for Goodcall’s clients. With a high volume of calls coming in every day, labeling massive amounts of data with high-quality annotations is a time-consuming process. To address this challenge, Goodcall began working with Scale Rapid to acquire high-quality text annotations. These annotations provide the data that software engineers need to evaluate and improve their models, while also freeing up time and resources at the company.

The Problem

Goodcall Had Too Much Data to Annotate In-house

Before working with Scale Rapid, Goodcall had mountains of real-world production data that was left unannotated. Because they were performing data annotation in-house, they couldn’t match the scale of available data. Every piece of unlabeled data was one less example that Goodcall could use to improve their models.

By training their models on a larger, more diverse dataset, Goodcall could unlock opportunities for improving model performance. With additional data, they could represent a wider range of customer experiences and identify and address edge cases that frequently led to model failures. For example, Goodcall could recognize utterances that their models performed poorly on, and make adjustments to compensate, improving overall accuracy. To provide the best possible results for every customer call, Goodcall needed a scalable, sustainable approach for labeling large quantities of data.

"When a caller talks to a business, the ultimate goal is that their query is resolved properly. The Scale pipeline allows us to review our model’s performance, and improve that model on a call by call basis. We couldn’t do that before – At 125,000 calls a month, reviewing them ourselves isn’t feasible. Now, our engineering team can focus on the right problems."

Bob Summers

CEO and Founder

Goodcall

The Solution

Scale Rapid Provides Goodcall with High-quality Text Classification Annotations

Scale Rapid provides simple, user-friendly tools that make it easy for companies like Goodcall to acquire the high-quality annotations they need. Goodcall can easily send a large collection of samples to Scale Rapid to be reviewed. In return, they receive text classification labels that describe the meaning of certain phrases. For example, if a text sample were to say, “I want to know when this business closes,” Goodcall would receive a label back from Scale indicating that this phrase is requesting business hours.

Goodcall extracts even more value from the annotations they get from Scale Rapid by checking these human-reviewed labels against the unlabeled items in their system. This way, they can have a wider variety of data that they can use to improve their models. With these annotations, Goodcall can improve the accuracy of a model’s text-to-speech transcripts, as well as enhance a bot’s ability to correctly understand intent.

“Scale AI proved to be a valuable data annotation partner for our City Operations Team, providing us high quality of training and validation data that helped us significantly improve the accuracy of our image classification models.”

Greg Stewart

Central Operations Project Manage

Skip

The Result

Goodcall Provides an Even Better Customer Experience

Since Goodcall started working with Scale Rapid, they’ve improved their model’s intent classification by 5%. By outsourcing the labeling process, they’ve also freed up engineering time to tackle other problems. There have been additional unexpected benefits, too – Using Scale Rapid’s text classifications, Goodcall has identified opportunities for improvement in their speech recognition system, by uncovering limitations in their transcript and intent classification models.

These text labels provide Goodcall with valuable insight into their chatbots, allowing them to quantify their models, identify issues, and assess why their models perform the way they do. Armed with this information, they’re able to make intelligent decisions to improve their product and, by extension, the customer experience.

“When it comes to model improvement, every percentage point counts. Improving 100,000 calls by 1% means that 1,000 people are getting a better experience. Every percentage point is a victory, and with Scale Rapid, we’ve experienced 5% improvement over just 3 months. We’re taking an insurmountable task and making it surmountable.”