Scale

Annotation Attributes Overview

In many cases, it is useful to have more human-judged metadata on top of each annotation for a given task, for example, measuring the occlusion-level of all vehicles in an image.

To achieve this, we support

annotation_attributes
, an object representing additional attributes that you'd like to capture per image annotation.

You may use

annotation_attributes
to define categorical attributes, numerical attributes, angle attributes, text attributes and more for each annotation.

You define the type of the attribute using the

type
property of the attribute, if no type is specified, it will default to a
category
type attribute. See the Annotation Attribute Types below for more details.

The format for

annotation_attributes
is an object whose key-value pairs all specify attributes of each annotation that you want to capture. The schema differs slightly based on the type of attribute.

Annotation Attribute Types

Attribute Type

Description

Categorical

Multiple choice attribute, with an optional ability to enable selecting multiple options simultaneously.

Numerical

Integer input

Angle

Input to select a value between 0 and 360 with a visual interface supporting angles.

Text

Input for free text

X/Y Offset

Input to select width or height within a box annotation

Request Format

To create a task with attributes, simply add the

annotation_attributes
parameter to your task creation request using the format described above.

Example task payload for an `annotation` task

{
  "callback_url": "http://www.example.com/callback",
  "instruction": "Draw boxes around the vehicles in the image.",
  "attachment_type": "image",
  "attachment": "http://i.imgur.com/v4cBreD.jpg",
  "geometries": {
    "box": {
      "objects_to_annotate": ["car","pedestrian"],
    }
  }
  "annotation_attributes": {
    "parked": {
      "description": "Is the car currently parked?",
      "choices": [
        "Yes",
        "No"
      ]
    },
    "heading": {
      "description": "Which direction is the car heading",
      "choices": [
        "left",
        "right",
        "back",
        "front"
      ],
      "conditions": {
        "label_condition": {
          "label": "car"
        },
        "attribute_conditions": [
          {
            "parked": "No"
          }
        ]
      }
    }
  }
}

Response Format

In addition to the standard attributes for the annotation response, if you specified

annotation_attributes
in the request, each annotation object will contain an
attributes
JSON. It will contain keys for each of the attributes you defined in the
annotation_attributes
schema, and the values will be the categorical value or numerical value chosen by a labeler.

📘 See the Callback Section for more details about callbacks.

Example response with attributes is below:

{
  "response": {
    "annotations": [
      {
        "left": 123,
        "top": 10,
        "width": 121,
        "height": 39,
        "label": "car",
        "attributes": {
          "parked": "Yes",
          "heading": "left"
        }
      },
      {
        "left": 82,
        "top": 56,
        "width": 64,
        "height": 30,
        "label": "car",
        "attributes": {
          "parked": "No",
          "heading": "front"
        }
      },
      { ... },
      { ... }
    ]
  },
  "task_id": "5774cc78b01249ab09f089dd",
  "task": {
    // populated task for convenience
    ...
  }
}

Numerical Attributes

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • type
      : a string describing the type of attribute. Must be
      number
      for numerical attributes.

    • min
      (optional): The minimum number that may be chosen as the attribute.

    • max
      (optional): The maximum number that may be chosen as the attribute.

    • step
      (optional, default
      1
      ): The legal number intervals for the attribute.

    • conditions
      (optional): a JSON that describes the conditions under which this attribute should be collected.

Example of a numerical attribute.

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "example_number": {
      "type": "number",
      "description": "What is the answer to this example question?",
      "min": 0,
      "max": 100
    }
  }
}

Categorical Attributes

Each key-value defining a categorical attribute should have the following structure:

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • choices
      : a list of strings or LabelDescription objects corresponding to the categorical choices.

    • type
      (default
      category
      ): a string describing the type of attribute. Must be
      category
      for categorical attributes.

    • conditions
      (optional): a JSON that describes the conditions under which this attribute should be collected.

    • allow_multiple
      (optional): a boolean value, if
      true
      , allows multiple values to be selected. The response will be an array of values.

Request Example

Example of a categorical attribute.

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "example_attribute": {
      "type": "category",
      "description": "What is the answer to this example question?",
      "choices": [
        "Answer1",
        "Answer2"
      ],
      "allow_multiple": true
    }
  }
 ...
}

Response Examples

Response Examples

{
  "example_attribute": "Answer1"
}
{
  "example_attribute": [
    "Answer1",
    "Answer2"
  ]
}

Angle Attributes

An angle attribute allows you to receive some angular information of the attribute, such as the heading yaw angle or heading pitch angle. This will return a number from

0
to
360
, which is the value of the angle in degrees.

Each key-value defining an angle attribute should have the following structure:

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • type
      : a string describing the type of attribute. Must be
      angle
      for angle attributes.

    • conditions
      (optional): a JSON that describes the conditions under which this attribute should be collected.

Example of a angle attribute.

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "example_angle": {
      "type": "angle",
      "description": "What is the angle of this object?",
    }
  }
}

Text Attributes

A text attribute allows you to receive freeform text, such as a transcription of a road sign or a car's license plate. This will return a string.

Each key-value defining a text attribute should have the following structure:

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • type
      : a string describing the type of attribute. Must be
      text
      for text attributes.

    • conditions
      (optional): a JSON that describes the conditions under which this attribute should be collected.

Example of a text attribute.

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "example_text": {
      "type": "text",
      "description": "What does this sign say?",
    }
  }
}

X/Y Offset Attributes

X/Y offset attributes are supported for

box
and
cuboid
geometries and represent some x- or y-offset within the annotation. They may be used to mark key positions such as the center of the annotated object.

In visual representations of task responses, they will be rendered as a horizontal/vertical lines across the annotation.

Each key-value defining an X/Y offset attribute should have the following structure:

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • type
      :
      "x_line"
      or
      "y_line"

Example definition of an X/Y offset attribute:

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "center": {
      "type": "x_line",
      "description": "Where is the center of the vehicle?",
    }
  }
}

Example response with X/Y offset attributs:

{
  "response": {
    "annotations": [
      {
        "left": 123,
        "top": 10,
        "width": 121,
        "height": 39,
        "label": "car",
        "attributes": {
          "center": 60
        }
      }
    ]
  }
}

Linked Attributes

Linked attributes are used to link one annotation in a task to another annotation - for example, specifying the car that a tire belongs to.

Each key-value defining a linked attribute should have the following structure:

  • key: a short string that serves as the identifier for the attribute

  • value: an object that should have the following keys:

    • description
      : a human-readable string describing the attribute to a labeler.

    • allowed_labels
      : a list of strings corresponding to the label choices this object can be linked to. If using Label Nesting, the strings in
      allowed_labels
      can only be the final "leaf" nodes.

    • type
      :
      linked

    • required
      : For
      imageannotation
      ,
      segmentannotation
      ,
      videoannotation
      , and
      videoplaybackannotation
      tasks, a boolean value representing whether this attribute is required to be annotated. Defaults to
      false
      . Note that for 3d Sensor Fusion Tasks, this option is not supported. For Sensor Fusion tasks, specifying a
      linked
      attribute will require an object to be linked before a task can be returned to you.

    • conditions
      (optional): a JSON that describes the conditions under which this attribute should be collected. See Conditionality for more details.

If the linked attribute has a value, the value of the attribute will be the annotation UUID of the object it was linked to. If the linked attribute is not used, the linked attribute will not be included in the response at all.

Example of a linked attribute.

{
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "example_linked_attribute": {
      "type": "linked",
      "description": "What is this object connected to?",
      "allowed_labels": [
        "Truck",
        "Car"
      ],
      "required": true
    }
  }
 ...
}

Example of Linked Attribute Response

// This example shows two polygons representing friends. 
// The first has no linked friends. The second example is linked to the first. 

{
  "annotations": [
    {
      "label": "Friend",
      "uuid": "a6092706-266d-4de1-8a14-fa6c682a57ea",
      "vertices": [
        {
          "x": 360.73736572265625,
          "y": 200.52745056152344
        },
        {
          "x": 365.7757263183594,
          "y": 348.65576171875
        },
        {
          "x": 510.88104248046875,
          "y": 295.24896240234375
        },
        {
          "x": 468.55865478515625,
          "y": 176.34324645996094
        }
      ],
      "geometry": "polygon"
    },
    {
      "label": "Friend",
      "attributes": {
        "friend": "a6092706-266d-4de1-8a14-fa6c682a57ea"
      },
      "uuid": "c6e00dc4-a3c0-4d48-9203-8f02d9c76328",
      "vertices": [
        {
          "x": 619.7100219726562,
          "y": 166.2664794921875
        },
        {
          "x": 443.36676025390625,
          "y": 255.94961547851562
        },
        {
          "x": 507.8580017089844,
          "y": 367.8016052246094
        },
        {
          "x": 693.2703247070312,
          "y": 313.3871154785156
        }
      ],
      "geometry": "polygon"
    }
  ]
}

Global Attributes

Using the

"is_global": true
flag in the
conditions
JSON when defining attributes marks the attribute as global.

For

imageannotation
and
segmentannotation
tasks, this means the attribute applies to the scene as a whole instead of any particular annotation. Therefore, the
label_condition
condition will never apply.

For

videoannotation
and
videoplaybackannotation
tasks, this means the attribute applies to
events
which occur on a frame or during a group of frames. The
label_condition
can be used to restrict the attribute to appear on specific event labels.

For

imageannotation
tasks, the global attribute
image_rotation
will sync with the image rotation in the task.

Example attribute specification for image annotation and segment annotation tasks

{
  "annotation_attributes": {
    "is_night": {
      "type": "category",
      "description": "Does this scene take place at night?",
      "choices": ["Yes", "No"],
      "conditions": {
        "is_global": true
      }
    }
  }
}

Conditionality

The

conditions
JSON describes the set of
conditions
under which the attribute should be collected. Each of these conditions objects described in the JSON are
OR
'd together, and each property within these objects is
AND
'd together.

Definition:
Conditions

The

conditions
object describes which labels and attributes to ask for based on already provided responses.
conditions
can have up to two key-values:
label_condition
and
attribute_conditions
.

Definition:
Condition

A

Condition
is an object defined by a single key and a value. The key describes the semantic string that you want to check, whether it is
'label'
for the label_condition or the attribute identifier for
attribute_conditions
.

The value is either a string or an array of strings. In the case of a string, the label or attribute will be checked for strict equality with the string. In the case of an array of strings, the label or attribute will be checked for membership within the array.

Condition Parameters

The

label_condition
key-value will describe the label or labels that you want to collect this attribute for. The value is a
Condition
object, which must have a key equal to
'label'
and a value corresponding to the label or furthest "leaf" node / subchoice of a
LabelDescription
object.

For example, if in the above example, you wanted an "occlusion" attribute for some mix of the "leaf" labels, you could specify the label condition for the attribute as e.g.

{ "label": ["Car", "Truck", "Motorcycle", "Adult"] }
. If you wanted the attribute to be conditional on all the subchoices of a given label (e.g., all "Vehicle"s), you would simply specify all of those subchoices in the array,
e.g.
{ "label": ["Car", "Truck", "Train", "Motorcycle"] }
.

The

attribute_conditions
key-value will describe under what conditions with the other attributes you'd like to collect this attribute for. The value should be a
Condition
object or a list of
Condition
objects.

attribute_conditions
are automatically verified for circular dependencies, and tasks will be rejected if any circular dependencies are found.

Example `conditions` parameter

"conditions": {
  "label_condition": {
  	"label": ["Car", "Truck", "Motorcycle"]
  },
  "attribute_conditions": [{ "is_moving": "No" }, { "is_parked": "Yes" }]
}

Example `Condition` objects

// condition for the label being equal to vehicle
{ "label": "vehicle" }

// condition for the label to be equal to either car or truck
{ "label": ["car", "truck"] } 

// condition for the is_parked attribute to be equal to be Yes
{ "is_parked": "Yes" }

Example of Conditions

{
  ...,
  "geometries": {
     ...
  },
  "annotation_attributes": {
    "occlusion": {
      "description": "What percent of the object is occluded?",
      "choices": ["0%", "25%", "50%", "75%"],
      "conditions": {
        "label_condition": {
        	"label": ["Car", "Truck", "Motorcycle"]
        }
      }
    }
  },
  ...,
}

Global Track Attributes (Multiframe Tasks)

For

videoannotation
,
videoplaybackannotation
, and
lidarannotation
tasks, using the
"global": true
flag when defining attributes marks an attribute as a global track attribute. This means the attribute's value will be constant across the entire object track, instead of being allowed to vary from frame to frame.

Multiple Types of "Global" Attributes

Global Attributes (here) are different from Global Track Attributes for Multiframe Tasks.

Global Attributes specify a global, scene-level attribute, or an attribute of events if using events in

videoannotation
or
videoplaybackannotation
tasks. It is not used to customize the attributes of a single annotation.

Global Track Attributes (Multiframe Tasks) specify one attribute of annotations when using multiframe tasks, like

videoannotation
or
videoplaybackannotation
. Once set, all frames of the annotation must have the same, consistent attribute value. Global Track Attributes should only be used to specify annotation attributes of
videoannotation
,
videoplaybackannotation
or
lidarannotation
tasks.

Example attribute specification for a global attribute

{
  "annotation_attributes": {
    "car_color": {
      "type": "category",
      "description": "What is the color of the car?",
      "choices": ["red", "green", "blue"],
      "global": true,
    }
  }
}
Updated 3 months ago