Updating Your Project Setup
After launching a batch, there are some aspects of your project setup that can be updated and others that are locked:
Project Setup | Can be updated after launching a batch |
|---|---|
Uploaded data | Yes, you can upload and archive data at any time. |
Use case | No |
Pipeline settings | Coming soon! For now, you can reach out to a developer success engineer to help with the migration. |
Advanced settings | Yes, you can access these in “Settings” on your Batches page |
Taxonomy | Yes, but note that this comes with some degree of complexity. Please refer below for more on how to do this. |
Instructions | Yes, you can update your instructions. Please refer below for more on how to do this. |
Taxonomy
Updating your taxonomy after onboarding requires a few followup actions in order to keep your project in a healthy state. Click on ‘Task Setup’ in the sidebar navigation in order to preview your current taxonomy and instructions. In the top right corner, click on ‘Edit Taxonomy’ to get to the taxonomy editor. As a reminder, the changes made in this flow will not update any live tasks and will only be applied to your next batch.
Make the necessary changes here (refer to the Taxonomy section of our Project Creation workflow for help with this editor) and hit next. This will bring you to the ‘Propagate Changes’ step, where you can update all of the tasks that are affected by your taxonomy changes. The first section shows you a revision history as it pertains to the affected tasks (note: changes made via the JSON editor will not show up in this section). Review this and confirm that it is accurate.
In the next three sections, you will see galleries of instruction examples, training tasks, and quality tasks. Here you can push your new taxonomy version to the tasks individually or all at once. We will try our best to reconcile the task responses according to your changes. While flipping through tasks, confirm that the task responses match what you expect and create/delete any annotations that were reconciled improperly. Note that we are not able to generate any annotations for the new labels that you’ve added to your taxonomy so you will have to complete this yourself.
Updating your taxonomy after onboarding requires a few followup actions in order to keep your project in a healthy state. Click on ‘Task Setup’ in the sidebar navigation in order to preview your current taxonomy and instructions. In the top right corner, click on ‘Edit Taxonomy’ to get to the taxonomy editor. As a reminder, the changes made in this flow will not update any live tasks and will only be applied to your next batch.
Make the necessary changes here (refer to the Taxonomy section of our Project Creation workflow for help with this editor) and hit next. This will bring you to the ‘Propagate Changes’ step, where you can update all of the tasks that are affected by your taxonomy changes. The first section shows you a revision history as it pertains to the affected tasks (note: changes made via the JSON editor will not show up in this section). Review this and confirm that it is accurate.
In the next three sections, you will see galleries of instruction examples, training tasks, and quality tasks. Here you can push your new taxonomy version to the tasks individually or all at once. We will try our best to reconcile the task responses according to your changes. While flipping through tasks, confirm that the task responses match what you expect and create/delete any annotations that were reconciled improperly. Note that we are not able to generate any annotations for the new labels that you’ve added to your taxonomy so you will have to complete this yourself.
Once you’re satisfied that all of your examples, training tasks, and quality tasks reflect the new taxonomy, proceed to the final step of updating your instructions. At this point, your new taxonomy has already been saved and any task updates have been propagated. Here we provide suggestions based on your changes (i.e. highlighting a section that was renamed, creating a section for new labels, recommending to delete any obsolete sections). Review our suggestions, revise if necessary, and confirm via the stepper. This will take you back to taxonomy and instructions previews where we conclude this process. Your changes will be applied to the next batch.
Instructions
Iterating on your instructions is an important part of the calibration process. We often find that our most successful users revise their instructions after every completed batch (adding edge case examples, tweaking phrasing, clarifying terminology). Writing clear, thorough instructions is the easiest and most direct way to increase your annotation quality. Once your first calibration batch is complete, you can review the instructions feedback for each task on the batch page. This feedback will also be aggregated on the instructions editor view underneath the instruction outline.
When you are ready to edit your instructions, head over to the task setup page from the sidebar navigation. Here you can preview both your existing taxonomy and instructions. In the top right corner, click ‘Edit Instructions’ to enter the instructions editor. Make any changes as you see fit, and click ‘Update’. This will provide you with 2 options: updating live tasks with this new version of your instructions or deferring it to subsequent batches. Select the option that matches your intent and click ‘Confirm’.