experiment

Header

Customers

Discover how leading companies leverage Scale's AI solutions to transform their operations and drive innovation. Explore success stories and insights.

Leading Brands Trust Us | Scale

LogoCloud

Footer

PreFooter

SectionTitle

NextImage

BoxCustomer

AnnouncementContextProvider

Screen

plasmic-graphcms

scale.com (Production)

lottie-react

plasmic-nav

react-scroll-parallax-global

plasmic-embed-css

plasmic-query

plasmic-basic-components

plasmic-tabs

react-aria

plasmic-cms

react-chartjs-2

plasmic-strapi

react-slick

radix-ui

react-quill

university logo-horizontal_BLK.png

Logotype Black.png

GCP_139x24px 1.svg

Group 58.png

Frame 66.png

Frame 65.png

Frame 64.png

launch code?.png

Frame 62.png

Frame 63.png

USEUCOM.svg

US_Indo-Pacific_Command_Seal.svg

Logos - Google.svg

Anduril.svg

scale logo.png

legal_new.svg

time_logo.svg

leaderboard.png

center.svg

aicpa-soc-logo-freelogovectors.net_ 3.svg

iso-2-1 1.svg

fedramp-logo-vert 2.svg

ntt_logo.svg

cengage_logo.svg.svg

dla-piper 1.svg

cisco.svg

Logos - Valeo.svg

50-AI-logo.svg

bcg.svg

Black.png

Logos - Global Atlantic.svg

Howard_Hughes_logo.png

4DV Analytics - png.png

Logos - Uncountable.svg

Logos - Grata.svg

stability ai.svg

Logos Antrhopic.svg

Logos - Brex.svg

image.jpg

openai.svg

Logos - Nuro.svg

kaleido_ai_black.svg

Luminar Logo White Horizontal Vector 4.svg

time_lockup.png

TIME Vignette: Using POTY Data Mark--small.mp4

TIME is disrupting the media industry with GenAI experiences like TIME AI for TIME’s Person of the Year. In our latest case study, hear from Jessica Sibley (CEO, TIME) and Mark Howard (COO, TIME) describe how Scale delivered this impactful and trustworthy GenAI experience in just two months.

Disrupting the media industry with TIME AI.

Visit us at Scale.com

time_lockup (2).png

TIMELogo.png

TIME and Scale revolutionize media publishing by leveraging the power of Generative AI to bring TIME&rsquo;s trusted journalism to the world in exciting new ways. Together we built TIME AI, the first interactive Generative AI experience for TIME Person of the Year.&nbsp;
The TIME AI experience includes text and audio summaries, voice interactions, translations, audio, and conversational chat, enabling readers to immerse themselves in the history and background of the Person of the Year. Custom guardrails keep the model on topic and safe for use.

TIME is Redefining the Media and Publishing Industry with Generative AI

timecasestudythumb.png

The media industry faces an existential risk due to the rise of Generative AI technology. As media consumption preferences are changing, TIME wanted to lean in and push the boundaries of what it meant to engage with their content.&nbsp;
They needed to move past a traditional article's static, linear experience and even beyond an out-of-the-box conversational chat experience. They needed a solution that was unique to TIME and that reflected the organization's high quality standards and their foundation of trusted journalism.&nbsp;
TIME wanted to build an AI solution that could guide readers deeper into articles, enhance comprehension, maintain their editorial voice, and operate safely&mdash;all while blocking inappropriate content.&nbsp;
TIME also needed this solution to be safe to maintain their foundation of trusted journalism. TIME person of the Year is usually highly contentious and the subject of lively debate.
Finally, TIME needed a partner that was capable of accelerating from proposal to production solution in just two months.

TIME Needed to Quickly Build an Immersive and Trustworthy AI Solution

With TIME AI, TIME has extended its audience, expanded accessibility, and increased engagement. But this project was about more than just driving engagement&mdash;it was about staying relevant in a changing media landscape. By providing dynamic, interactive, multimodal experiences that are personalizable and customizable, TIME positioned itself as a disruptive leader.
The innovative partnership between TIME and Scale garnered attention not just for the Person of the Year content, but for the groundbreaking way it was delivered. This strengthened TIME's position as a forward-thinking, relevant media brand at the intersection of trusted journalism and cutting-edge technology.
Looking ahead, TIME and Scale are building even more ambitious and disruptive experiences across their digital footprint.

TIME Redefines How People Experience and Engage with their Trusted Journalism

Scale designed and built a dynamic, interactive, and multimodal solution to highlight the 2024 TIME Person of the Year Donald Trump, and the winners from the three previous years, Taylor Swift, Volodymyr Zelensky, and Elon Musk.
Text and audio summaries make it easier to consume the articles, while voice input and conversational chat add depth to the experience.&nbsp;
The articles were translated into seven languages: English, Spanish, French, German, Chinese, Russian, and Ukrainian. The conversational chat experience is contextually aware of the Person of the Year articles and other related articles from TIME&rsquo;s archives.
Readers can ask TIME AI questions like: "What are Donald Trump&rsquo;s plans for foreign policy?" or "How do Taylor Swift&rsquo;s record sales compare to other successful musicians in the 20th and 21st century?" or "Has the Russia - Ukraine border been disputed in the past?". They can also catch up on past editions by listening to articles in a podcast format while multitasking or receive concise summaries focused on the subjects within the content that matter most to them.
Implementation
Scale integrated fine-tuned models from its partner OpenAI to provide the chat, summarization, and translation capabilities, and ElevenLabs for audio functionality. Scale fine-tuned these models on TIME&rsquo;s vast archives of proprietary articles.
Scale implemented comprehensive guardrails for the system by drawing on extensive previous red-teaming and AI safety work done for the U.S. Government and Fortune 500 enterprises.
To tune the TIME AI guardrails, Scale used in-house red-teaming to establish and enforce strict safety standards. Over 7,000 attack vectors were tested to protect against AI misuse and custom-tailored attacks.
Red-teaming is essential in the AI development lifecycle, serving as a safeguard to proactively identify vulnerabilities, biases, and harmful behaviors that could arise under unexpected conditions. By constantly iterating and refining the model's responses through these techniques, we ensure that the model is resilient against a wide spectrum of adversarial inputs and tailored to TIME's editorial voice.
Scale delivered these features in less than two months from proposal to production deployment. The Scale team accelerated the development of this application with Scale GenAI Platform (SGP),&nbsp; using building blocks for fine-tuning, frontier evaluations, red-teaming, guardrails, monitoring, and more.

Scale Delivers a Unique and Trustworthy AI Solution in Two Months

Customer Success Story: TIME | Scale AI

Cohere works with Scale to improve its Command model with high-quality prompt/response pairs. 

Cohere Enhances its Large Language Models with Scale.

Cohere

C-Blog-Post-4x-pink--1-.png

Cohere is a natural language processing (NLP) startup whose aim is to make NLP useful and accessible to anyone who needs it. Cohere&rsquo;s platform allows users to generate, categorize, and organize text at a scale that was previously unimaginable by developing large language models that easily integrate with existing systems.

Cohere is Building the Future of Language AI

The team at Cohere had an urgent need to acquire high-quality data to train the first iteration of their instruction-following model, Command. However, the Cohere team, &ldquo;needed to build up an annotation force quickly, and Scale provided access to a large-scale, diverse, and skilled workforce to employ for this complex data creation job,&rdquo; explained Alex Wang, Technical Lead at Cohere. The team at Cohere knew they needed a partner who could provide diverse and complex inputs (prompts) and outputs (responses) to ensure their models would perform well on challenging and unusual queries their users may have.

Cohere Needed High-Quality Data For Their Instruction-Following Model.

The data provided by Scale accounted for a sizable portion of Cohere&rsquo;s first Command dataset. By collaborating on instruction-following best practices and doubling down on complex, tightly paired data, Scale contributed to performance gains for Cohere&rsquo;s Command model. Command is one of the highest-performing LLMs across <a href="https://crfm.stanford.edu/helm/latest/?group=core_scenarios" target="_blank" rel="noopener noreferrer">HELM</a> and other standardized metrics. Cohere retrains its LLMs on a weekly basis to constantly improve the models, and that requires a regular stream of high-quality data.
&nbsp;
&ldquo;Scale has been incredibly responsive to our data needs, both at a macro level (designing novel workflows for collecting complex types of data) and a micro level (adjusting existing data collection protocols to better align the data with our needs). We give feedback to Scale weekly, and they are always receptive to our feedback, positive or negative. If we request a change, they are quick to move and have the change implemented within a couple of weeks. We feel like valued customers at Scale,&rdquo; concluded Wang.

Cohere's Command LLM Improves with High-Quality Data

Large Language Models (LLMs) are typically developed in 3 stages:
<ol>
<li>Stage 1: Training with a large corpus of raw data&nbsp;</li>
<li>Stage 2: Instruction fine-tuning using prompt/response pairs</li>
<li>Stage 3: Reinforcement learning with human feedback (RLHF)&nbsp;</li>
</ol>
Scale and Cohere initially partnered on stage 2: instruction fine-tuning using prompt/response pairs. Partnering with Scale solved both of Cohere&rsquo;s challenges. &ldquo;They have a skilled workforce capable of delivering high-quality prompt/response pairs and their engineers worked closely with us to develop the necessary tooling and infrastructure. All we had to do was communicate our needs with Scale. This greatly freed up our engineering time and resources to tackle other urgent problems while still acquiring the critical data we needed,&rdquo; said Wang. Scale was also able to cover a wide distribution of questions to provide a greater lift in model performance.

Scale's Generative AI Data Engine Delivers High-Quality Prompt/Response Pairs.

Customer Success Story: Cohere | Scale AI

Developing accounts payable training data with Scale Document AI

Logos - SAP.svg

SAP is a leading software corporation headquartered in Walldorf, Germany, that develops enterprise software for the &ldquo;Management of business processes, developing solutions that facilitate effective data processing.&rdquo; Known best for its enterprise resource planning (ERP) software, SAP aims to help companies and organizations of all sizes run their businesses profitably, adapt continuously, and grow sustainably. &ldquo;We have a rich spectrum of use cases ranging from document processing, analytics, master data matching, process automation, and many more,&rdquo; said Urko Sanchez, Senior Director, Head of AI Functions, Europe. &ldquo;For this collaboration with Scale, we focused on improving our products around document processing, especially those dealing with invoices, purchase orders, and payments advices,&rdquo; he added. &ldquo;By bringing AI-powered products and services, we are helping companies to modernize and become future-proof,&rdquo; he concluded.

Scale Document AI for Accounts Payable

The team at SAP had a trove of customer documents but needed a partner to create a comprehensive dataset to enhance their accounts payable products&nbsp;while respecting the data ownership, privacy, and sensitivity of its customers. High-quality data is critical for performant models. SAP needed higher quality training data to train models for processing and extracting crucial information from purchase orders and invoices in English, German, and Spanish. &ldquo;Our customers are very heterogeneous. We have customers that provide us with thousands of documents a week, while others might require months for a fraction of the same volume. Sometimes, customers might not even know in advance what amount of data they have available,&rdquo; explained Sanchez. &ldquo;Therefore, finding a reliable and flexible partner for creating high-quality training datasets was key for us,&rdquo; he added.

Document variability made high-quality training data a challenge

ML models learn associations between input and output data. If there are errors in training or validation data, the resulting model will underperform or learn the wrong task. By leveraging Scale, SAP was able to improve their Business Document Processing (BDP) services portfolio. &ldquo;Our services are now much more accurate and can deal with new types of documents and languages,&rdquo; explained Sanchez. &ldquo;Overall, we have experienced a great boost in accuracy across the board. The whole process has also been a valuable experience to understand better the trade-off between the cost of higher quality data and the accuracy of our models,&rdquo; concluded Sanchez.

Enhanced Business Document Processing Services

With deep expertise in labeling data across a wide range of use cases, the team at Scale delivered high-quality labeled data across three languages, multiple document types, and 200+ unique fields. Adding to the complexity, fields in documents that contained personally identifiable information (PII) such as names, phone numbers, and emails need to be replaced with semantically similar but different examples to preserve data privacy. Despite these challenges, Scale delivered labeled data at near-perfect accuracy on a quick ramp to high volumes.
&ldquo;We decided to go with Scale because we needed a heavily technical partner that could deal with changing requirements dynamically, guaranteeing availability, quality, and scale,&rdquo; said Sanchez. The Document AI team did this by partnering closely with SAP to understand the technical and business requirements of SAP&rsquo;s Business Document Processing (BDP) model, then leveraging a combination of machine learning-powered pre-labeling (ML pre-labeling) and Scale&rsquo;s global labeling operations to deliver high-quality data.

Data extraction at over 95% accuracy from diverse document types under 60 seconds

Optimus Ride focuses on rider safety for robotaxis

Developing safe and reliable systems with high-quality 3D training data

Optimus Ride is a Boston-based company that specializes in developing autonomous vehicles and systems for geo-fenced environments. Optimus Ride partners with local communities around the country to develop custom mobility systems capable of transporting people and goods safely and reliably. Optimus Ride’s team has focused on Geo-fenced environments to-date including mixed-use developments, city zones, office and industrial parks, ports, airports, academic campuses, retirement communities, and more.“The team is constantly developing cutting-edge algorithms to develop safe and reliable systems,” explained Michaela Mann, Product Marketing Manager at Optimus Ride. With this hyperlocal strategy and cutting-edge technology, Optimus Ride brings together the promise of self-driving technologies with real-world considerations to enable safe, efficient, and sustainable mobility networks.

Optimus Ride partners with local communities around the country to develop custom mobility systems capable of transporting people and goods safely and reliably

The Optimus Ride team initially labeled their data in-house. &ldquo;Unfortunately, it just wasn&rsquo;t as quick as we had hoped,&rdquo; explained Nuno Almeida, Team Lead and Manager of AI, Optimus Ride. As Optimus Ride expanded into new environments and worked on developing more novel perception algorithms, they collected more data than they could effectively label in-house. &ldquo;We needed a partner who could more efficiently and quickly label all this data we&rsquo;re collecting,&rdquo; added Almeida.

More data than they could effectively label in-house

"We're focused on making sure we can bring all of the wonderful technology that exists today to our customers and our clients. We&rsquo;ve deployed across four states and our nation&rsquo;s capital so far and we work really closely with and listen to the needs of our clients to adapt with them and provide different product offerings that will allow us to continue expanding,&rdquo; explained David Robert, Head of Product at Optimus Ride. &ldquo;We're going to continue to increase our deployments, and the training data Scale AI provides will ensure that what we&rsquo;re doing actually has practical use and enduring value for people,&rdquo; he concluded.

Adapting with customers and providing product offerings that can scale

The Optimus Ride team selected Scale AI as its data annotation provider for a few reasons; &ldquo;With Scale AI, the main thing was that we can get our data labeled more quickly and at a higher quality than doing it in-house,&rdquo; noted Almeida. Second, &ldquo;We really liked all the SLAs that were provided to us and the whole Scale AI team stepped up and worked with us to meet our needs,&rdquo; said Mann. &ldquo;Scale&rsquo;s commitment to quality has been really amazing. We&rsquo;ve sent over different schemas and different attributes several times. Each time, we were able to align quickly and easily address any issues as they arose,&rdquo; noted Kushagra &ldquo;Kush&rdquo; Tiwary, Perception Engineer, at Optimus Ride. &ldquo;The hardest part of perception is all the edge cases. Having a partner that can flag these and address them swiftly has been great,&rdquo; Tiwary added.
Lastly, &ldquo;From an engineering point of view, the APIs that Scale provides allows us to create a system to automate all the interactions between our systems and Scale&rsquo;s systems,&rdquo; said Almeida. &ldquo;All the logging functions allow us to easily check for errors and help with the whole process of quality assurance,&rdquo; Almeida concluded.

Labeled data more quickly and at higher quality

Customer Success Story: Optimus Ride | Scale AI

Goodcall uses Scale Rapid's text-classification data labeling and annotations to improve their chatbots

Goodcall Boosts Call Quality with Scale Rapid

Goodcall

Goodcall provides businesses with intelligent phone agents so that their customers can always get in touch. When customers call these bots to request information like the business&rsquo;s hours, location, and products or services, these phone agents perform automatic speech recognition (ASR) to convert speech-to-text. Then, once the text is acquired, the bots employ AI analysis to interpret the person&rsquo;s request and to provide the appropriate response. The intent classification models that power Goodcall&rsquo;s chatbots need to be regularly fine-tuned with real-world production data. Any performance improvement in the model also means an improvement in the customer experience for Goodcall&rsquo;s clients. With a high volume of calls coming in every day, labeling massive amounts of data with high-quality annotations is a time-consuming process. To address this challenge, Goodcall began working with Scale Rapid to acquire high-quality text annotations. These annotations provide the data that software engineers need to evaluate and improve their models, while also freeing up time and resources at the company.

Goodcall Leverages Scale Rapid to Improve their State-of-the-art Chatbots

Before working with Scale Rapid, Goodcall had mountains of real-world production data that was left unannotated. Because they were performing data annotation in-house, they couldn&rsquo;t match the scale of available data. Every piece of unlabeled data was one less example that Goodcall could use to improve their models.
By training their models on a larger, more diverse dataset, Goodcall could unlock opportunities for improving model performance. With additional data, they could represent a wider range of customer experiences and identify and address edge cases that frequently led to model failures. For example, Goodcall could recognize utterances that their models performed poorly on, and make adjustments to compensate, improving overall accuracy. To provide the best possible results for every customer call, Goodcall needed a scalable, sustainable approach for labeling large quantities of data.

Goodcall Had Too Much Data to Annotate In-house

Since Goodcall started working with Scale Rapid, they&rsquo;ve improved their model&rsquo;s intent classification by 5%. By outsourcing the labeling process, they&rsquo;ve also freed up engineering time to tackle other problems. There have been additional unexpected benefits, too &ndash; Using Scale Rapid&rsquo;s text classifications, Goodcall has identified opportunities for improvement in their speech recognition system, by uncovering limitations in their transcript and intent classification models.&nbsp;
These text labels provide Goodcall with valuable insight into their chatbots, allowing them to quantify their models, identify issues, and assess why their models perform the way they do. Armed with this information, they&rsquo;re able to make intelligent decisions to improve their product and, by extension, the customer experience.

Goodcall Provides an Even Better Customer Experience

Scale Rapid provides simple, user-friendly tools that make it easy for companies like Goodcall to acquire the high-quality annotations they need. Goodcall can easily send a large collection of samples to Scale Rapid to be reviewed. In return, they receive text classification labels that describe the meaning of certain phrases. For example, if a text sample were to say, &ldquo;I want to know when this business closes,&rdquo; Goodcall would receive a label back from Scale indicating that this phrase is requesting business hours.
Goodcall extracts even more value from the annotations they get from Scale Rapid by checking these human-reviewed labels against the unlabeled items in their system. This way, they can have a wider variety of data that they can use to improve their models. With these annotations, Goodcall can improve the accuracy of a model&rsquo;s text-to-speech transcripts, as well as enhance a bot&rsquo;s ability to correctly understand intent.

Scale Rapid Provides Goodcall with High-quality Text Classification Annotations

Customer Success Story: Goodcall | Scale AI

Scaling ground-truth data with Scale AI. Classifying large volumes of images to develop an autonomous checkout system

Standard Cognition strives to improve everyone’s retail shopping experience by making shopping a seamless and more human interaction. Standard does this by developing an autonomous checkout platform for brick and mortar retailers that can change how people shop in the future. Their approach to autonomous checkout is unique because of their vision-first approach that identifies customers by shape and movement, rather than by facial recognition, and customers can shop in a Standard-powered store with or without their mobile app.

About Standard Cognition

&ldquo;Our system was very naive because it was still very early stage,&rdquo; said Virginia Puccio, Standard&rsquo;s Head of Data Operations. To improve the performance of their deep learning models, Standard's team required a large and varied dataset. Despite this need, &ldquo;we didn&rsquo;t have a workforce to handle the volume of annotation tasks, nor did we have the tools,&rdquo; she added. Puccio and the Standard team knew they needed a scalable data labeling solution.

Improving the performance of models

Those in the San Francisco Bay Area can experience the future of shopping at Standard&rsquo;s flagship store in downtown San Francisco. As Standard expands its operations globally, more and more shoppers will be able to experience this new paradigm for themselves.

Experience the future of retail

The Standard Team selected Scale for their expertise in working with large data sets. &ldquo;It was easy to get started with Scale AI because they had the platform and the people with the skillset to perform the annotation tasks we needed,&rdquo; noted Puccio. &ldquo;While we&rsquo;re much bigger now and could potentially handle the volume of annotation tasks, why would we want to? ... We believe we should leave data annotation to experts like Scale so we can focus on perfecting the retail shopping experience with regard to the use of AI,&rdquo; Puccio concluded.
&nbsp;
By working with Scale, the Standard team have been able to focus on other aspects of their business, such as the collection of data and offering more support services to their customers.

Perfecting the retail shopping experience

Customer Success Story: Standard Cognition | Scale AI

At Harvard Medical School, the Datta Lab accelerates their research using Scale Rapid with data labeling and annotation.

Harvard Medical School's Datta Lab Accelerates their Research Using Scale Rapid

Harvard Medical School

The Datta Lab at Harvard Medical School is studying the neural mechanisms associated with behavior in rodents. To conduct their research, they place mice in an experimental rig, and then record each mouse&rsquo;s behavior using cameras. They also measure each mouse&rsquo;s neural activity using neural implants. They can then analyze this data to examine the link between neural activity and behavior, as well as to advance their understanding of specific brain areas of mice. Actually analyzing this data, however, is a time-consuming process &ndash; To interpret the behavioral data, the researchers must first label the poses of the mouse over time. While the lab can train machine learning (ML) models to perform this automatically, a large amount of video footage needs to be manually annotated first. This annotation process detracts from the time that researchers could instead be spending on the aspects of research that require their expertise. The Datta Lab at Harvard Medical School is now working with Scale Rapid to get the annotations they need. Instead of spending weeks labeling their data, they can pour their full attention into advancing their research.

With Scale Rapid Annotating their Data, the Researchers in the Datta Lab Can Focus on the Science

wdcgTEwzgKk-HD.jpg

Analyzing a mouse&rsquo;s behavioral data requires annotators to take the raw video of a mouse and extract detailed kinematic variables that describe its body&rsquo;s position at each moment in time. In particular, researchers are often interested in the mouse&rsquo;s joint position over time, such as the location of its toes, knees, and hips.
Before researchers can train a model to detect these key points, however, they first need to build a training set. This is not a quick process &ndash; An annotator needs to sit in front of a computer for hours labeling hundreds of images. Before the Datta Lab worked with Scale Rapid, this annotation was performed entirely in lab. This consumed a huge amount of time that could otherwise have been spent on the more salient aspects of research.

The Datta Lab Needed to Speed Up their Data Annotation Process

Instead of spending weeks annotating data, the lab now sends their data to Scale Rapid and gets quality annotations as quickly as overnight. The easy access to data annotations has lowered the barrier to training new models, so the lab has been able to try out new ideas and iterate more rapidly. Now, if a researcher needs to collect new training data after updating an experiment, or if they need to track a different part of the mouse in existing data, they can easily send their footage to Scale Rapid for new annotations. This makes it possible for the lab to explore possibilities that simply weren&rsquo;t feasible before.
Additionally, the ease with which the lab can obtain high volumes of data annotations has made it easier to produce high-performing ML models. Having a large amount of data can reduce the need for model fine-tuning &ndash; The majority of edge cases are naturally handled by the model, so researchers don&rsquo;t need to spend weeks adjusting their model for these shortcomings. By using Scale Rapid, the Datta Lab can now spend more of their time advancing their research in neuroscience.

With Scale Rapid, the Datta Lab gets Better Research Outcomes Faster

Now that the Datta Lab is working with Scale Rapid, they no longer need to annotate their data in-house. Instead, they can send their raw videos of mice to Scale Rapid and quickly get quality annotations in return. These annotations provide the lab with the information they need to train their ML models for automated pose tracking.
Obtaining annotations from Scale Rapid saves the lab weeks of labeling time, allowing researchers to spend their time more effectively. Working with Scale Rapid also allows the lab to acquire a higher volume of annotations than they could ever accomplish in-house. Having a large volume of high quality annotations makes it easier than ever for the lab to train high-performing ML models, while also freeing up researcher time.

Scale Rapid Quickly Provides the Datta Lab with the Annotations They Need to Advance their Research

Customer Success Story: Harvard Medical School | Scale AI

Kaleido AI asked Scale to help them generate synthetic data to help them continue to improve their model performance on object identification and improve the IoU of their model predictions. Kaleido AI had plateaued at an IoU of 0.657 with real-world data and needed to find a way to improve its model continuously.

Scale’s Synthetic Data Helps Kaleido AI Bring Better Visual AI to Everyone

Kaleido AI

kaleido_ai_color.svg

Headquartered in Vienna, Austria, Kaleido AI&rsquo;s mission is to make complicated tech simple. By creating tools that simplify and accelerate workflows, foster creativity, and enable others to bring new ideas to life, the company provides everyone - from individuals to businesses of all sizes - access to the latest advancements in visual AI.
In 2019, Kaleido AI introduced remove.bg, the automatic image background remover, built to eliminate manual work and complex tools with high learning curves. In 2020, the team launched Unscreen, an online software that removes video backgrounds in no time. These simple but effective tools dramatically increased the speed at which users could achieve their goals. As a result, their popularity skyrocketed, which ultimately led to Canva acquiring Kaleido AI in early 2021.
Later that year, Kaleido AI launched <a href="https://www.designify.com/?utm_source=partnership&amp;utm_medium=blog&amp;utm_campaign=synthetic_case_study">Designify</a>, the AI-powered tool that creates automatic designs for individuals, car dealerships, e-commerce websites, and more.
Kaleido AI&rsquo;s incredibly effective and accurate tools use machine learning algorithms to automatically detect objects and backgrounds and return background-free photos and videos. Through easy-to-use APIs and plugins, developers and editors accelerate their workflows using Premiere Pro, After Effects, or other editing software.
Kaleido AI uses Scale Synthetic for synthetic data generation, edge case identification, and Scale&rsquo;s machine learning expertise.

Synthetic Data is Key for Improving Kaleido AI's Models

Kaleido AI takes an AI-first approach to its visual editing tools, with a talented team of ML engineers on staff. Enabling simple, automated visual editing tools that work requires excellent machine learning models and excellent machine learning models need a large volume of high-quality data.
However, Kaleido AI encountered several edge cases in a specific segmentation task where their model performed poorly. Collecting and labeling tens of thousands of real-world images with a large diversity of patterns, images, backgrounds, and textures was difficult. Open datasets did not have enough high-quality images of this particular class.
In the initial attempt to solve this issue, Kaleido AI relied on real-world data to train its segmentation models. Unfortunately, collecting and labeling a large amount of diverse real-world data was complex, resource-intensive, and costly. After this initial effort, the team determined that instead of spending the time and resources to develop their models with real-world data, they should consider using synthetic data to augment their dataset. So, they sought help from the experts in synthetic data at Scale AI.&nbsp;

Kaleido AI Needed Better Data to Improve Segmentation for Challenging Edge Cases

Today, Kaleido AI&rsquo;s model is built on five hundred real-world images and almost fifteen thousand synthetic images from Scale. Kaleido AI needed to improve its model performance and efficiency to enable visual AI for everyone. Scale Synthetic helps Kaleido AI achieve its goal by enabling continuous and efficient model performance improvement on target edge cases.
Looking forward, Kaleido AI intends to continue to increase the amount of synthetic data in its dataset, using Scale&rsquo;s Synthetically generated data to improve its models. There are many use cases that Kaleido AI is considering focusing on next, and Kaleido AI will continue to look to Scale to help them improve their model performance.
With Scale Synthetic, Kaleido AI has reduced the time and effort involved with synthetic data curation, so the team can now focus on improving its core application.

With Scale Synthetic, Kaleido AI Improves Model Performance While Focusing on Building Their Core Product

Kaleido AI asked Scale to help them generate synthetic data to help them continue to improve their model performance on object identification and improve the IoU of their model predictions. Kaleido AI had plateaued at an IoU of 0.657 with real-world data and needed to improve its model continuously.
Scale&rsquo;s ML Engineers started by analyzing Kaleido AI's sample data and model inferences in Nucleus, Scale&rsquo;s data curation platform. Scale quickly identified that the model was performing poorly in segmenting objects in images with complex patterns, the objects were shaded or transparent, or where there were shadows in the backgrounds of the scenes. Scale focused on these edge cases and generated a sample of 2,650 images of synthetic data with varied lighting, textures, and patterns. However, this first pass was not sufficient to meaningfully improve model performance.
The Scale team recognized that they needed more data specifically targeted at problem edge cases to improve model performance substantially. So the team did a deep dive into <a href="https://scale.com/nucleus" target="_blank" rel="noopener noreferrer">Nucleus</a> to curate data to further identify these problem edge cases. They also introduced the ability to visualize Scale&rsquo;s synthetic images compared to real images in 2D space. To do this, the Scale team first passed all the images to a RESNET 101 backbone and took the vector embedding of each image at various layers before passing them to t-SNE. Then, the team visualized which synthetic images were furthest in 2-dimensional space from real images and which real images formed a distinct and unaddressed cluster of images. Using this method, the Scale team identified the real images furthest from synthetic images and prioritized these edge cases.
This analysis revealed that Scale needed to include more textured/patterned objects and a wider variety of object types in the synthetic data distribution.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fb1568c7d117f4044baaab8b8be3c1154">
Upon further investigation, the clump of red images towards the center of the plot were patterned objects.&nbsp;
This analysis identified additional patterns and other edge cases on which to focus. We also determined that white objects on white backgrounds with dirt or speckled backgrounds or where there are two layers of background (i.e., rooms with doors) were causing poor model performance.
The Scale team now had a clear understanding of what was needed to improve model performance and set about generating a larger dataset with data targeted at addressing these edge cases. In total, Scale generated 14,583 synthetic images covering a total of 12 categories covering patterns, various objects, backgrounds, and textures. With this targeted synthetic data, Kaleido AI achieved an IoU of 0.794.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fc9b63e6ed407451bbe168b3add1c6f66">

Synthetic Helps Kaleido AI’s ML Team Prioritize and Generate Better Data to Improve Model Performance on Business-Critical Edge Cases

Customer Success Story: Kaleido AI | Scale AI

OpenSea detects digital marketplace deception with Scale Content Understanding.

Copymint Prevention for NFT Marketplaces

OpenSea

OpenSea is on a mission to build an open digital economy, helping the world’s creators, collectors and collaborators own and shape their relationships directly. As the first and largest marketplace for non-fungible tokens (NFTs), customers can browse, buy, sell, and mint NFTs across seven different blockchains on the OpenSea platform.

OpenSea is the World’s Leading Web3 Marketplace

Trust and safety are some of the most significant barriers to welcoming new people into the Web3 ecosystem. While Web3 and NFT technology are starting to mature, users new to the world of NFTs are still often deceived by fake content and need help properly distinguishing between original NFTs and copymints (or NFTs that are duplicates or imitations of popular NFTs). As a leader in the space, OpenSea was looking for a vendor to help advance their detection and removal capabilities to identify and mitigate copymints and fraud as early as possible.&nbsp;

Detecting Digital Marketplace Deception

After kicking off the project with Scale Content Understanding, OpenSea was able to dramatically accelerate their copymint detection capabilities &ndash; today detecting copyminted NFTs in real time. Scale provided model-based systems to help process large volumes of data and provide signals to the OpenSea team. One of the critical measures of success for OpenSea was reducing the latency between an NFT being created on the platform to identifying bad content and taking it down.&nbsp;
The second measure of success is the ability to handle the large volumes of data from OpenSea. Scale processes up to 50 million items a week with 95% average precision. By quickly detecting and removing inauthentic NFTs, OpenSea is able to improve user trust in their marketplace. Here&rsquo;s to a safer OpenSea, and thus a safer web3.

Decreased Time to Deceptive NFT Takedown 

OpenSea needed an industry-leading solution that could identify and handle a dynamic set of deceptive NFTs. Before working with Scale, the OpenSea team was early in their AI journey. Though the team already used rule-based systems to help capture forms of deception, it was a challenge to attain the desired speed, recall, and precision needed to effectively address fraud in the marketplace. OpenSea approached Scale due to the team&rsquo;s experience building customized models and ability to ramp quickly for customers.&nbsp;
<a href="https://scale.com/content-understanding" target="_blank" rel="noopener noreferrer">Scale Content Understanding</a> provides data enhancement for better platform experiences by enriching, analyzing, and categorizing content. Here, Scale Content Understanding proved robust in testing against OpenSea&rsquo;s high data volume of up to 50 million items a week and proved an effective partner for tackling their problems.
With Scale Content Understanding, the team offered a fast turnaround models-as-a-service solution to categorize and determine whether a given NFT is a close match of another one. Scale&rsquo;s ML models provided multiple layers of deduplication processes: starting with real-time detection through API-based solutions and full catalog scans to remove historical scams.
Real-Time NFT Detection
When an NFT is minted, OpenSea needs to quickly detect whether or not it is a copymint and remove it from its site to mitigate the chance of a user purchasing that copyminted item. Scale Content Understanding facilitates this through two ways: 1) a real-time API and 2) a recurring batch job:
<ol>
<li>The real-time API receives live traffic from items minted on OpenSea&rsquo;s platform and items ingested via the block chain, scans the items through Scale&rsquo;s ML model, and returns the likelihood that it is a copymint. All of this occurs within a matter of seconds.</li>
<li>The recurring batch job runs on an hourly cadence to pick up any items that may have fallen through the real-time API.&nbsp;</li>
</ol>
In order to satisfy the scale and complexity of the problem, the Scale team trained custom deep learning image models to represent NFTs as embeddings in a manifold. With this setup, similar items will be nearby, meaning copymints would sit near each other, and items that are significantly different will have a further distance from one another. Scale converted and stored all these NFT embeddings in a vector database and built out systems for real-time querying and retrieval through k-Nearest Neighbors algorithms.
&ldquo;The real-time detection of fuzzy matches is really tricky for systems to get right, but I think Scale&rsquo;s models have really nailed it.&rdquo; - Charles Zaffaroni, Product Manager, OpenSea
Full Catalog Scans
Over time, OpenSea's operations team verifies more collections on their platform, and Scale's engineering team improves the model&rsquo;s performance based on this feedback. These improvements are retroactively applied to the complete set of items to take down any previously missed fraudulent items. Scale Content Understanding provides the capability to run full catalog scans, scanning hundreds of millions of items with high precision over a short period of time.

Real-Time Fraudulent NFT Detection on OpenSea’s Platform 

Customer Success Story: OpenSea | Scale AI

Research and development of next-generation autonomous systems that can safely operate in a range of work environments

OTTO Motors is an Ontario-based company that specializes in developing flexible and intelligent mobile robots for automated materials handling in manufacturing facilities and warehouses around the world. OTTO’s industrial-grade platform moves goods intelligently through facilities when and where they are needed, streamlining production while safely avoiding pedestrians. OTTO robots operate safely and efficiently in a wide variety of environments. “One of our biggest initiatives is to make sure our robots are reliable and accurate; ensuring they function effectively in the various environments where they are being deployed,” said James Servos, Autonomy Engineering Manager at OTTO Motors.

About OTTO Motors

To research and develop next-generation systems that can better understand, navigate, and safely operate in a range of work environments, the OTTO team needed data from a variety of diverse environments. At the time the OTTO team first began working on this problem, most publicly available datasets for autonomous mobility were specifically for autonomous road vehicles like cars and trucks. &ldquo;For what we wanted to do, there was no publicly available data,&rdquo; said Nolan Lunscher, Senior Autonomy Developer at OTTO. &ldquo;We had to start from scratch on getting datasets and we needed a partner who could supply us with a steady supply of data in a domain that was quite different from what everyone else was working on,&rdquo; he added.

Developing Datasets from Scratch

By working with Scale AI, Servos and his team are able to focus on their actual work of improving its systems rather than on data annotation. &ldquo;Someone like [Lunscher] is an incredibly high-value worker as an autonomy engineer. Working with Scale means keeping him focused on developing cutting edge algorithms and state of the art networks,&rdquo; Servos noted.
&nbsp;
The diversity of environments and areas OTTO&rsquo;s systems must operate in is vast. &ldquo;Every time we deploy to a new customer site, we find new corner cases. We&rsquo;ll continue to refine our systems and push more data through Scale&rsquo;s labeling pipeline to develop generalized networks that can function in all of these locations,&rdquo; Servos concluded.

The Future of Warehousing and Manufacturing

The OTTO team selected Scale AI as its data annotation provider for a few reasons. First, Scale AI&rsquo;s ability to annotate a range of data types from 3D sensor data to camera imagery gave the OTTO team confidence in Scale AI to handle the kinds of data they had. Second, Scale AI&rsquo;s experience in the autonomous driving space meant Scale AI could help the OTTO team better understand possible edge cases. &ldquo;The On-Demand Service also gave us a good opportunity to trial the system to get a feel for how the process works and how the tools work,&rdquo; said Lunscher. &ldquo;Once we had a feel for it, we were able to jump in with our larger project,&rdquo; he noted.
&nbsp;
&ldquo;The cost of developing all of the tools and workflows to annotate data for a company like us was just completely not feasible,&rdquo; Servos added. &ldquo;The Scale AI team was able to annotate our data at a pace far above what we would have expected while keeping the overhead for data annotation reasonable,&rdquo; he concluded.

Throughput and Volume OTTO Can Count On

Customer Success Story: OTTO Motors | Scale AI

Making drones useful

To accomplish this goal, Skydio is building the easiest-to-fly, hardest-to-crash drone featuring a state-of-the-art subject tracking and obstacle avoidance system that allows Skydio’s drones to follow its customers through the woods, over the mountains, and across the sea.

Skydio’s goal is simple: to make drones useful by making them smart

To ensure their subject tracking and obstacle avoidance system functions for a wide variety of scenarios and locations, Skydio&rsquo;s autonomy team, led by Hayk Martiros, needed a large and varied dataset to train its models with. In order to focus on actually developing its models and drones, the Skydio team knew they needed a data labeling partner they could rely on.

Finding the right partner for accurate data annotation

With the release of the Skydio 2, customers can see and experience for themselves the most advanced drone on the market.

The Skydio 2

Skydio looked into various solutions for its data annotation needs, but chose Scale AI for our product features, flexibility and team. &ldquo;The team and infrastructure Scale AI had ready to go let us focus on using our data instead of labeling it,&rdquo; noted Martiros. Scale AI&rsquo;s team made itself a valuable annotation partner by being responsive and continually working to improve its products.
&ldquo;When we presented them with an unusual labeling need, they added a feature to do it,&rdquo; said Rachel Basse. &ldquo;We appreciate how the Scale AI team proactively seeks feedback and holds themselves accountable. Their QA team is careful and quick to seek clarification, and their attention to detail has produced some truly impressive results,&rdquo; she added. They readily adapt with us when unforeseen complications arise in our data. We can implement small changes to requirements with a short conversation or roll out new projects in a few weeks, with minimal development time from us,&rdquo; Martiros concluded.

Products

Enterprise

Government

Resources

Customers

Leaderboards →

Customers

Disrupting the media industry with TIME AI.

Cohere Enhances its Large Language Models with Scale.

Developing accounts payable training data with Scale Document AI

Developing safe and reliable systems with high-quality 3D training data

Goodcall Boosts Call Quality with Scale Rapid

Scaling ground-truth data with Scale AI. Classifying large volumes of images to develop an autonomous checkout system

Harvard Medical School's Datta Lab Accelerates their Research Using Scale Rapid

Scale’s Synthetic Data Helps Kaleido AI Bring Better Visual AI to Everyone

Copymint Prevention for NFT Marketplaces

Research and development of next-generation autonomous systems that can safely operate in a range of work environments

Making drones useful

Annotating large volumes of videos to develop advanced control technology for workboats and other commercial vessels

Safely handling autonomous trucking edge cases with synthetic data.

Powering automated bill pay with Scale Document AI

Processing mission-critical logistics documents with Scale Document AI

Velodyne uses Scale Nucleus to find edge cases in 3D data and curate the most valuable data for annotation

Delivering a Seamless Cellar Management Solution

Enhancing Log Scaling and Inventory Management with Scale Rapid

Nuro mines for rare classes in unlabeled data with Nucleus Object Autotag.

Scale Rapid Helps Orchard Robotics Deliver State-of-the-Art Precision Crop Management

With Scale Studio, Vistapath Enhances Patient Experience.

Machine intelligence for faster and safer real estate transactions

Pietra Enhances E-Commerce Product Imagery with Scale.

Enhancing Pick-and-Place Robots with Annotations from Scale Rapid

Building In-house Labeling Operations for High-Quality Training Data

With Scale Rapid, advanced.farm Scales its Apple-Picking Operations

The future of your industry starts here.