At Scale, we are excited to help you solve your most challenging computer
vision problems. One important and well-studied problem within computer vision
is that of semantic segmentation, which aims to understand images at the pixel
level. More precisely, in semantic segmentation, we want to understand which
semantic class each pixel of an image belongs to. Below we show a street
scene, semantically segmented into a small set of class labels including
vehicles, pedestrians, and road markings:
A street image
The image segmented into semantic classes
Supervised approaches trained on large datasets of pixel-wise annotations are
currently the best at predicting semantic classes in several public benchmarks
and challenges. Unfortunately, obtaining accurate pixel-wise training data can
be incredibly time-consuming; each pixel of an image must be manually assigned
a label by a human, usually through the use of Photoshop-like tools.
To provide our customers with the fastest turnaround times for semantic
segmentation, Scale has added a “Guided Automatic Segmentation” feature to our
segment annotation tool. We hope to make the process of creating pixel
annotations faster and easier by allowing our labelers to provide only a few
points of a segment and automatically filling in the rest.
Obtaining segment annotations by clicking a few key boundary points!
Segmenting a car with four extremal points as human input.
road label with four extremal points as human input.
Our new semi-supervised tool allows us to empower our labelers with insights
from current data-driven methods. Instead of attempting to segment the entire
image from scratch, our labelers must only click on at least four extremal
points of any object. By reducing our area of interest from the whole picture
to a smaller single-class semantic region, we can use the convolutional
features from the image data augmented by the chosen extremal points to create
a proposed pixel map, which can then be further fine-tuned at the labeler’s
discretion.
Our Guided Automatic Segmentation tool ultimately allows our workers to label
pixel-wise annotations much faster with higher quality, so we can continue to
label all of your semantic segmentation data at best-in-class accuracy and
cost. We are excited to continue exploring other ML techniques to improve all
of our APIs -- stay tuned for more blog posts coming soon!