Write your instructions

High quality instructions unlock high quality labels! We recommend investing time into writing clear and comprehensive instructions in order to ensure labelers are aligned with your labeling goals.

After creating your taxonomy, we auto-generate an instructions outline that you can populate. We highly recommend that you fill in every section that is generated:

• Summary: Provide labelers an introduction to your task. You can also use this section to give any other context that would be useful for the labeler when annotating your data. For example, what type of scenery will they see? How many frames will the labeler see in each task? What types of objects will they need to look for? What is unusual about this task?

• Workflow: In this section, you should write a step by step guide on how to complete a task. Some ideas to consider: What should the labeler notice first about the task? What sort of deductive reasoning may the labeler encounter? Will a single annotation affect any other annotations?

• Rules: You can use the Rules section to describe annotation rules that apply to multiple labels or attributes. For each rule, you will be prompted to fill out a Description to describe what the rule is and you can apply the rule to different labels. You will also be prompted to add well-labeled and poorly labeled examples.

• Label / Attribute / Field: You can use the label-/attribute-/field-specific sections to describe rules that uniquely apply to the label/attribute/field. Again, you will be prompted to fill out a Description and to add well-labeled and poorly labeled examples.

Adding examples to the Rule, Label, and Attribute sections: You can add various example blocks to separate different groups of examples from each other under the same annotation rule. Each example block includes a description for the block of examples and two columns for well-labeled examples and poorly labeled examples. You can use the side-by-side nature of the examples to help highlight differences between correct and incorrect labeling.

Each column can contain multiple examples. When you click on “Add another example,” you can either select an asset that you have uploaded already or upload a new asset. Then, you can label the task and mark it as either a well-labeled or poorly labeled example.

You can also create instructions examples while you’re auditing. Read here for more details.

Writing instructions is very important! We highly recommend that you dedicate time to write good instructions in order to achieve high quality on the platform. For more tips, read more here.