Below are some best practices to consider as you write instructions:

1. Dive into your data: Before writing instructions, it’s important to review a representative sample of your data to understand what global and edge case rules need to be written. As you’re reviewing the data, put yourself in the labeler's shoes; document areas you’d need guidance on, as well as any tricky cases that need to be addressed in the instructions.

2. Writing instructions:

   • Structure: Following a clear structure in your instructions will help labelers better understand the rules, leading to higher quality annotations. The below structure is a good starting point:
   • Summary of Task: Provide a few pithy bullets describing the basics of your task.
   • Workflow: Provide the steps labelers must take to annotate your task from start to finish.
   • Annotation Rules: Start with the global rules (i.e. rules that apply to all tasks, then zoom in to more specific rules). Areas you’ll likely want to touch on:
     • Minimum pixel size
     • What to label and what NOT to label
     • How to manage occlusion, truncation, and low visibility cases
     • Geometry sizing if relevant
     • How to manage cases you’re not sure about (i.e. “err on the side of selecting XXX”)
     • Etc.
   • Label and Attribute Definitions and Examples
   • Common Errors
   • Edge Cases

3. Communication:

   • Communicate succinctly and clearly (ie pithy bullets, use spaces to decrease cognitive load for the labeler)
   • Include at least 1 image for each point you’re looking to make. Make sure each image has a description of WHY it is correctly or incorrectly annotated.

4. Pressure test your instructions: After writing the instructions, review your data again and make sure your instructions cover the majority (90+% of cases you see). Continue to iterate on instructions until you have hit the 90+% number.

5. Final tips:

   • It can be helpful to include a short video walking through the instructions
   • If there are certain tools / features that you expect labelers to use when annotating your data, we’d suggest mentioning them in the instructions and providing a description of how to use the tool (e.g. interpolation)