At Scale, we are excited to help you solve your most challenging computer

vision problems. One important and well-studied problem within computer vision

is that of semantic segmentation, which aims to understand images at the pixel

level. More precisely, in semantic segmentation, we want to understand which

semantic class each pixel of an image belongs to. Below we show a street

scene, semantically segmented into a small set of class labels including

vehicles, pedestrians, and road markings:

A street image

The image segmented into semantic classes

Supervised approaches trained on large datasets of pixel-wise annotations are

currently the best at predicting semantic classes in several public benchmarks

and challenges. Unfortunately, obtaining accurate pixel-wise training data can

be incredibly time-consuming; each pixel of an image must be manually assigned

a label by a human, usually through the use of Photoshop-like tools.

To provide our customers with the fastest turnaround times for semantic

segmentation, Scale has added a “Guided Automatic Segmentation” feature to our

segment annotation tool. We hope to make the process of creating pixel

annotations faster and easier by allowing our labelers to provide only a few

points of a segment and automatically filling in the rest.

Obtaining segment annotations by clicking a few key boundary points!

Segmenting a car with four extremal points as human input.

road label with four extremal points as human input.

Our new semi-supervised tool allows us to empower our labelers with insights

from current data-driven methods. Instead of attempting to segment the entire

image from scratch, our labelers must only click on at least four extremal

points of any object. By reducing our area of interest from the whole picture

to a smaller single-class semantic region, we can use the convolutional

features from the image data augmented by the chosen extremal points to create

a proposed pixel map, which can then be further fine-tuned at the labeler’s

discretion.

Our Guided Automatic Segmentation tool ultimately allows our workers to label

pixel-wise annotations much faster with higher quality, so we can continue to

label all of your semantic segmentation data at best-in-class accuracy and

cost. We are excited to continue exploring other ML techniques to improve all

of our APIs -- stay tuned for more blog posts coming soon!