The Quality tab allows you to audit a random sample of tasks for quality and view stats for tasks you’ve audited
Why should you audit tasks?
Auditing tasks allow you to provide feedback directly to our taskers. This will help us identify common errors or clarify edge cases
There are two ways to audit tasks:
- Audit completed tasks individually
- Audit all completed tasks within a date range
- Audit all tasks completed across a specific week
How to audit completed tasks individually:
- In the Tasks tab, filter on the Completed tasks
- Search for the task that you want to audit and click on "Audit Task"
How to audit all completed tasks within a date range:
- In the Quality tab, select the specified task date range, audit or batch status
- Click on the "Audit Tasks" button - this will allow you to audit a random sample of tasks
Audit all tasks completed across a week:
- In the Quality tab go to the "Tasks completed last week" tab under "Report"
- Select Audit for the desired week
If you have multiple people on your team auditing at the same time, we will “lock” each task so that only one person can audit one task at a time. This was designed to prevent conflicted copies.
Within the audit workflow, you are able to make the following selections:
- Approve: Mark that tasks that have been completed according to instructions
- Make Changes: Fix tasks that have minor or non-critical errors that you can use to update your callback response immediately
- Reject: Signal that tasks have completed incorrectly. Please provide detailed comments for how our team can fix these mistakes moving forward
Reject vs. Make Changes
- Both Reject and Make Changes will provide feedback to the Scale team about the quality of your tasks.
- For quick or minor changes, please Make Changes as you will get an updated callback response right away. In addition, not all rejected tasks will be resubmitted. Please contact your engagement manager if you want to discuss re-dos
- For critical and more structural misunderstanding of the instructions, please Reject the task and provide detailed feedback! This will help our taskers to improve.
- The quality assurance dashboard will display the total number of reviews, accepts, and rejects using the set of filtered tasks. In addition, we also provide the following reports:
- Tasks completed last week - lets you audit tasks that have been aggregated by completed date from Monday 00:00 to Sunday 23:59 PST for the last 4 weeks
- Task Level Report - provides you with a list of tasks with auditor and completion time
- Auditor Tracker - shows an audit report for tasks in batches
- Note: this is only available if you are use batches. To learn more about batches, please see this article
- Auditor Quality - displays a summary of reviews, accepts, and rejects per auditor
When an audit is submitted, several new field will be added to the callback body:
customer_review_status: Reflects the result of the audit as “Accepted”, “Rejected” or “Fixed”
customer_review_comments: Shows all comment left during the audit
prior_responses: Response before the audit
If a task is fixed through multiple audits:
response: will be updated with the most recent audit
customer_review_status: will reflect the most recent audit
customer_review_comments: will append the comment to the array
prior_responses: will append previous responses to the array