Overview
Goodcall Leverages Scale Rapid to Improve their State-of-the-art Chatbots
Goodcall provides businesses with intelligent phone agents so that their customers can always get in touch. When customers call these bots to request information like the business’s hours, location, and products or services, these phone agents perform automatic speech recognition (ASR) to convert speech-to-text. Then, once the text is acquired, the bots employ AI analysis to interpret the person’s request and to provide the appropriate response. The intent classification models that power Goodcall’s chatbots need to be regularly fine-tuned with real-world production data. Any performance improvement in the model also means an improvement in the customer experience for Goodcall’s clients. With a high volume of calls coming in every day, labeling massive amounts of data with high-quality annotations is a time-consuming process. To address this challenge, Goodcall began working with Scale Rapid to acquire high-quality text annotations. These annotations provide the data that software engineers need to evaluate and improve their models, while also freeing up time and resources at the company.
The Problem
Goodcall Had Too Much Data to Annotate In-house
Before working with Scale Rapid, Goodcall had mountains of real-world production data that was left unannotated. Because they were performing data annotation in-house, they couldn’t match the scale of available data. Every piece of unlabeled data was one less example that Goodcall could use to improve their models.
By training their models on a larger, more diverse dataset, Goodcall could unlock opportunities for improving model performance. With additional data, they could represent a wider range of customer experiences and identify and address edge cases that frequently led to model failures. For example, Goodcall could recognize utterances that their models performed poorly on, and make adjustments to compensate, improving overall accuracy. To provide the best possible results for every customer call, Goodcall needed a scalable, sustainable approach for labeling large quantities of data.
The Solution
Scale Rapid Provides Goodcall with High-quality Text Classification Annotations
Scale Rapid provides simple, user-friendly tools that make it easy for companies like Goodcall to acquire the high-quality annotations they need. Goodcall can easily send a large collection of samples to Scale Rapid to be reviewed. In return, they receive text classification labels that describe the meaning of certain phrases. For example, if a text sample were to say, “I want to know when this business closes,” Goodcall would receive a label back from Scale indicating that this phrase is requesting business hours.
Goodcall extracts even more value from the annotations they get from Scale Rapid by checking these human-reviewed labels against the unlabeled items in their system. This way, they can have a wider variety of data that they can use to improve their models. With these annotations, Goodcall can improve the accuracy of a model’s text-to-speech transcripts, as well as enhance a bot’s ability to correctly understand intent.
The Result
Goodcall Provides an Even Better Customer Experience
Since Goodcall started working with Scale Rapid, they’ve improved their model’s intent classification by 5%. By outsourcing the labeling process, they’ve also freed up engineering time to tackle other problems. There have been additional unexpected benefits, too – Using Scale Rapid’s text classifications, Goodcall has identified opportunities for improvement in their speech recognition system, by uncovering limitations in their transcript and intent classification models.
These text labels provide Goodcall with valuable insight into their chatbots, allowing them to quantify their models, identify issues, and assess why their models perform the way they do. Armed with this information, they’re able to make intelligent decisions to improve their product and, by extension, the customer experience.