Multi-Stage Overview

Multi-stage is our most advanced and flexible task type.

It is designed to handle complex full-scene labeling that spans multiple annotation types and modalities, and serves as a replacement for Scale’s legacy dependent tasks system (which requires using multiple tasks to fully label a scene).

On a multi-stage task, labeling is split up into multiple stages. Stages are organized in a pipeline such that they are dependent on one another, and can run sequentially or in parallel. As a task passes through a stage, Scale adds labels corresponding to taxonomy classes enabled for that stage. Once all stages are completed, the task is then finalized and delivered.

Blueprints

All multi-stage tasks are created with a blueprint, which is defined separately prior to task creation. Each task’s blueprint dictates its stages, stage dependencies (pipeline), their respective taxonomies, and the values of any stage-specific parameters.

Blueprints may also include conditional logic, so that certain stages are only enabled if specific criteria are met (e.g. if there is an attribute with a specific value in the prior stage).

Multi-stage projects may contain one or many blueprints, but only one blueprint can be “active” at any given time. New multi-stage tasks will use the project’s active blueprint by default, but may also use a different blueprint from the project if specified during task creation.

A list of all blueprints available for each multi-stage project is available in the Scale dashboard.

Dependent Tasks vs Multi-Stage Tasks

	Dependent Tasks	Multi-Stage Tasks
Task Creation	Multiple tasks per scene Complex payloads containing fully defined dependencies and taxonomies Error-prone, frequently requires manual backfills to fix payload errors	One task per scene Simple payload referencing a single pre-defined blueprint On-rails, payloads are much less likely to contain errors
Project Management	Many projects, one for each dependency Difficult to track all the different tasks across all projects for each scene	One project Each scene is labeled with just one task
Task Delivery	Annotations are delivered across multiple tasks, each with a separate response	Annotations are delivered in a single task with a single response (task-level delivery), or delivered separately for each stage (stage-level delivery)
Auditing	Multiple audits across dependent tasks to fully audit a scene Support for Standard audits only	One audit is sufficient to review all annotations in the scene Supports both Standard Audits (stage-level delivery only) and Fixless Audit

Task-level Delivery vs Stage-level Delivery

Each multi-stage project can be configured to use either task-level delivery or stage-level delivery.

With task-level delivery, a multi-stage task is delivered when all of its stages are completed. Customer auditors can create Fixless Audits, and all annotations from all stages will be available for review. Quality will be tracked at the task-level as well, covering all annotations across all stages.

With stage-level delivery, annotations from each stage are delivered as soon as the stage is completed (even if other stages are still in-progress). Customer auditors can create either Standard or Fixless Audits, which will be conducted separately for each stage. Quality will also be tracked separately for each stage, covering only the annotations added in that stage.