Quality Debugging in Rapid
There are a couple main indicators for quality that can easily be improved by understanding what the underlying issues are. The process of creating a well performing project can be highly iterative, so these scores are meant to be a way to keep a pulse on what can be improved.
Batch type | Quality indicator |
|---|---|
Calibration batches | - Calibration Score, which can be found in Batches > Calibration Batch |
Production batches | - Evaluation accuracy by task, which can be found in Quality Lab > Evaluation Tasks |
Generally speaking, iterating on your instructions to make them comprehensive, clear, and concise is the highest leverage thing you can do to achieve high quality. You can learn more here about some best practices to keep in mind when writing instructions.
You can also refer here to learn about how to improve your Calibration Score and here to learn about different tools on the Rapid platform that you can leverage to improve quality on your production batches. To better understand evaluation accuracy by task metrics, you can also refer here. We describe other quality mechanisms you can leverage on Rapid in our Improve General Quality and Using the Issues Queue sections.
Furthermore, we recently released a new beta feature Custom Courses with Rapid (beta), where you can create and distribute more targeted and customized training materials to labelers.