Steven Hao

At Scale, we are excited to help you solve your most challenging computer vision problems. One important and well-studied problem within computer vision is that of semantic segmentation, which aims to understand images at the pixel level. More precisely, in semantic segmentation, we want to understand which semantic class each pixel of an image belongs to. Below we show a street scene, semantically segmented into a small set of class labels including vehicles, pedestrians, and road markings:<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/b3f2baa7-c352-4499-b4db-f024cd404a00/public">A street image<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/39d76976-0384-47be-705e-55b47e10ee00/public">The image segmented into semantic classes Supervised approaches trained on large datasets of pixel-wise annotations are currently the best at predicting semantic classes in several public benchmarks and challenges. Unfortunately, obtaining accurate pixel-wise training data can be incredibly time-consuming; each pixel of an image must be manually assigned a label by a human, usually through the use of Photoshop-like tools. To provide our customers with the fastest turnaround times for semantic segmentation, Scale has added a “Guided Automatic Segmentation” feature to our segment annotation tool. We hope to make the process of creating pixel annotations faster and easier by allowing our labelers to provide only a few points of a segment and automatically filling in the rest. Obtaining segment annotations by clicking a few key boundary points! <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/9971da18-0d5a-4387-d732-2a126f343900/public"> Segmenting a car with four extremal points as human input. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/e132db34-0b30-4d48-982d-af66f1afd100/public"> road label with four extremal points as human input. Our new semi-supervised tool allows us to empower our labelers with insights from current data-driven methods. Instead of attempting to segment the entire image from scratch, our labelers must only click on at least four extremal points of any object. By reducing our area of interest from the whole picture to a smaller single-class semantic region, we can use the convolutional features from the image data augmented by the chosen extremal points to create a proposed pixel map, which can then be further fine-tuned at the labeler’s discretion. Our Guided Automatic Segmentation tool ultimately allows our workers to label pixel-wise annotations much faster with higher quality, so we can continue to label all of your semantic segmentation data at best-in-class accuracy and cost. We are excited to continue exploring other ML techniques to improve all of our APIs -- stay tuned for more blog posts coming soon!

Steven Hao

Introducing AI-Assisted Semantic Segmentation at Scale