experiment

Header

Footer

PreFooter

SectionTitle

PaginationButton

BlogPage

Trusted by world class companies, Scale delivers high quality training data for AI applications such as self-driving cars, mapping, AR/VR, robotics, and more.

AI & ML Blog | Scale AI

NextImage

AnnouncementContextProvider

Screen

scale.com (Production)

lottie-react

plasmic-nav

react-scroll-parallax-global

plasmic-embed-css

plasmic-query

plasmic-basic-components

plasmic-tabs

plasmic-cms

react-chartjs-2

plasmic-strapi

react-slick

radix-ui

react-quill

Product

Company

General

People

Engineering

Q: Tell us a little bit about yourself and your role at Scale.A: My name is Sasha, and I'm a Machine Learning Engineer here at Scale. I've been at Scale for about a year and eight months. Previously, I was doing a master's program in computer science at Stanford, where my concentration was in artificial intelligence.Within my career at Scale, I've worn a lot of different hats. I first started working on the Nucleus team, in a backend software engineering role when the team was very young. It was&nbsp;Scale’s newest product at that point, and I helped build out a lot of Python SDK interfaces, and the backend schema for a lot of new features, including Autotag, which is the edge case detection feature.&nbsp;I later transitioned to building out other new products, including Scale Validate, which is the model scenario testing and validation product area.Most recently, I've recently transitioned to our Machine Learning team full time, where I'm on the Content Understanding team. I'm basically helping to support automation in labeling projects associated with content categorization and curation type labeling projects.Q: Could you give us a little bit of detail about your educational background, and how that led you to join Scale?A: My undergrad degree was in applied mathematics. Entering college, I didn't think of myself as a coding person at all. But during the years I was at Stanford, computer science was something that a lot of the undergraduate body talked about, and the entry level computer science classes were really fun. During my junior and senior years in college, I started taking some entry level machine learning classes. The first thing I ever learned about was logistic regression, and how you assign importance or attribution to different input features&nbsp;to answer social science questions. Like, is it age, or education, that contributes more to this downstream factor?I thought that was really interesting, and really wanted to double down on that. So, that's what led me to stay for a fifth year master's, and specialize in AI. I worked as a research assistant briefly, in a sociology lab, where the objective of the sociology lab was to develop a dataset, documenting urban change. My job as a research assistant was to label data on AWS, Mechanical Turk. We had a lot of difficulty controlling quality, and getting annotations that were detailed enough to model the downstream tasks of trash detection and vandalism detection.&nbsp;So, I knew that data collection was a hair on fire problem for a lot of application-driven teams. When I interviewed at Scale, I felt like the mission, and the problems that the company was trying to solve, were very urgent and impactful.Q: What attracted you to working at a hyper growth startup?A: When I was in college, I think a lot of my peers were talking about going to work at a big tech company; that was the prestigious choice. It was safe; it was a known outcome. At first, I was really focused on that, and I did a lot of internships at big companies.&nbsp;And then I got this career advice my sophomore junior year of college, from a woman at Grace Hopper, and her advice was get on a rocket ship. Basically, the idea was, if you could just put yourself in a place where there were interesting problems to solve, some of those were going to come your way.I really took that to heart, and my impression of myself started to shift toward somebody who really wanted to tackle big challenges and wear a lot of hats in their job. So, I joined Scale under the hypothesis that I was going to have an opportunity to wear a lot more hats and serve a lot of different functions at a very lean company.&nbsp;As a result of having the opportunity to take on difficult challenges, I've grown my skills at a very quick rate. And I think I gained seniority and experience, at a rate that far exceeds any previous work opportunity I had in the past.&nbsp;Q: What's your advice to students, or other people, who are interested in careers in machine learning?A: First of all, I think it's an excellent career to be in, in that there are a lot of opportunities in the space right now. And the number of positions for machine learning engineers and machine learning-adjacent engineers, is growing really rapidly. I think there are also a lot of exciting developments, both on the research and applications side.One thing that fascinates me about machine learning is how closely it can relate to real-world problems. If you’re interested in sociology, there's probably a place for that type of modeling for you. Or, if you're interested in online advertising, there are a lot of different application areas to concentrate on.The other thing that I would recommend is really reading a lot. There are a lot of great resources online for practicing building deep learning models, whether it be through Kaggle challenges, or what have you. Also, a lot of research papers and academic work are publicly available.One thing that I felt intimidated about when I started in the field was, how much there was to know, and how little of it I did know, as somebody that was early in their career. And I think making a habit of reading papers, and reading about approaches or problems that I didn't work on directly on a day-to-day basis, really increased my confidence.Q: Has working at Scale shifted your perspective on AI or ML applications at all?A: Definitely. When I first started at Scale, what I was really interested in was machine learning modeling work, and the idea of experimentation–building different neural network architectures, and being very scientific about, whether A works better than B.&nbsp;One thing that's become really apparent, both working at Scale, and seeing the types of problems that our engineering teams face, but also working with Scale customers, is the importance of systems design within large scale machine learning systems. I’ve seen how difficult the problems of scalability, observability and overall reliability of machine learning systems are.&nbsp;I really think that that's where a lot of innovation in the next 10 years is going to come from; not necessarily the modeling techniques themselves, but more, how you might integrate those modeling techniques into a larger scale operational framework.Q: What do you like to do outside of work?A:&nbsp;I've recently gotten into cycling. And so, I spend a lot of my time outside, whether it's running or riding my bike in the Bay Area. I also love wheel pottery. So on the weekends, you'll find me at the pottery studio.

image.jpg

Sasha Harrison details her experience as a Machine Learning Engineer.

Life at Scale

Document AI is powered by machine learning (ML) models that process documents at high accuracy and speed. When these models are fine-tuned for a customer, they become even more accurate and regularly achieve near-perfect accuracy levels and do not require any human intervention. Scale engineers continuously work to improve these models by creating new tools to make sure Document AI is highly reliable for the most critical industries we serve. This blog will highlight one of those tools, a technique called “linting”, and how it helps make Document AI’s models more accurate.<h2>What is Linting?</h2>At Scale, linting is the practice of creating rules or developing machine learning models that automatically detect errors in training data used to create models. In the case of Document AI, we leverage two types of linters: simple, quick linters and more complex ML-based linters.&nbsp;Here is an example of a simple linter comparing field types and their responses within a document.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fb8b843651dad4ba48f5831c07b48c6f2" width="755">In the example above, the letter ‘O’ should not be in a numerical, monetary amount field and our field-type linter would flag this example for correction.Another example of the simple quick linters is a linter that enforces known business rules. For example, a linter can ensure an invoice subtotal is equal to the sum of all line item totals.<h3><img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F9e0a34e10d464195911c07b617ee27c7" width="574"></h3>In the example above, the subtotal on the right ($16) is inaccurate and our linter would flag this example for correction.These simple linters are helpful where straightforward rules can easily be applied. Simple linters, while useful, can be implemented by most. What’s more unique about Scale Document AI is how we go beyond the mainstream methods and tools in the document industry to achieve high-quality.&nbsp;Scenarios that require a higher level of capability or understanding are better suited to machine learning (ML)-powered linters instead of rule-based linters, which is Scale’s specialty. Our specialized ML linters are trained to perform specific tasks, such as learning to compare differences in a transcribed text and the visual representation of the text as it appears on the document’s page. In this example, when the linter finds a meaningful discrepancy, it automatically corrects it or flags the document for review by highly experienced and skilled reviewers. This human-in-the-loop review is optionally available based on customers’ needs and preferences.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F626b6d1309fa4dc28c28d6d75279abb6">In the example shown above, we trained an ML-based linter to understand the context to know that “Account #” on the bottom left refers to a bank account number as it is in a section with ACH instructions while “Account Number” on the top right is referring to a business account number. If “Account Number” in the example on the top right had been identified as a bank account number, the linter would flag this for correction.ML linters are inherently more difficult to implement than rule-based linters as they have the same challenges as developing any other machine learning models. Developing an ML model is a statistical process prone to variation. That variation can manifest as unreliability or as overfitting. For example, unreliability can lead to an ML model producing wildly different predictions on two slightly different documents. With overfitting, the model learns the detail and noise in the training data and fails to perform on new data. In the case of linting, the ML models are trained on data with errors to help identify and flag other errors. The data used for training linters is inherently not of the highest quality; therefore it’s common that while the ML linter can still make decent suggestions, the model itself is not the most performant. This requires our ML team to place more restrictions and guardrails, like ensuring the confidence threshold is set high to only flag errors and make suggestions on things the model is fairly certain about. This prevents the model from producing too many unreliable linter suggestions that lead to human reviewers simply ignoring or not trusting these suggestions.&nbsp;Developing and deploying ML linters is also an iterative process with a feedback loop. As we deploy ML linters for our document processing models, the document processing models improve upon being retrained with higher quality data. As the document processing models improve, it gives better first results as well as learning other errors to surface to the human reviewers.<h2>How Do Linters Improve Models?</h2>The power of machine learning to recognize patterns between input and output data is its greatest strength as well as its Achilles’ heel. If the training data contains errors, the model will learn the wrong task. Having the right linters in place increases the accuracy of machine learning models by ensuring the data used for training is of the highest quality.&nbsp;For Document AI, we began by designing simple linters to check field types such as monetary amounts, quantities, and ZIP codes. We then expanded beyond these simple linters and implement ML-based linters to check for complex fields such as addresses with varying formats that we would need to parse and enforce.&nbsp;To make things more concrete, let’s think about the use case of invoices. At first glance, it seems straightforward to recognize an invoice: they all list the products sold or the services rendered, the price of each, and how payment should be made. However, invoice layouts, details, and content vary widely from business to business. Some invoices are particularly formal, containing everything from company bank account numbers and line item descriptions to account manager names and contact information. On the other end of the spectrum, an informal email between longtime business partners can also serve as an invoice. The unstructured and ever-changing nature of invoices make it challenging to model comprehensively and accurately.When building Document AI’s machine learning model for processing invoices, using linters resulted in a 16.3% relative decrease in transcription errors in the training data. This relative decrease in transcription errors is crucial, as the misspelled text makes it impossible for our document parsing models to extract the right information. This linter also helped remove the need for an additional layer of human quality assurance (QA) review while still ensuring that training data is pristine.&nbsp;Training our models on the higher quality training data led to a 14.3% relative increase in our document processing model’s accuracy. This improvement translates to increased model generalizability and robustness, and more accurate models mean fewer manual reviews, providing additional value for our customers.&nbsp;Linters make Document AI more accurate and hence deliver higher quality models to our customers who rely on high accuracy document processing.<h2>Parting Thoughts</h2>Scale Document AI distills insights from your messy, changing document data and delivers fast, accurate models tailored to your business without the need for templates. If you’re interested in learning more, take a look at our <a href="https://www.youtube.com/watch?v=jWCwVcQCA10" rel="noopener noreferrer" target="_blank">product video</a> or visit our <a href="https://scale.com/document-ai" rel="noopener noreferrer" target="_blank">product page</a>.Acknowledgments:We thank Malcolm Greaves from Scale’s Document AI Machine Learning team for his experiments that laid the groundwork for this post and for frequent consultations throughout the writing of this blog.

Document AI's linters improve model quality by ensuring common and hard-to-spot errors never make it into the training data.

Scale Document AI leverages ML Linting to Improve Document Processing

How ML Linting Meaningfully Improves Adaptive Document Processing at Scale

Willow leads the Customer Operations team at Scale. She is a former director of logistics at Amazon.com, a graduate of MIT’s Leaders for Global Operations program, and a former captain in the U.S. Army’s Military Police Corps. I am a transgender pioneer within the military and corporate world, with a career spanning 12 years in the U.S. Army, MIT, Amazon, and Scale. Here are some of the things I learned along the way. Why I decided to change my gender I had decided to change my gender while in a concrete shelter from rocket attacks right before returning from deployment in Afghanistan. Rockets in this context were not particularly threatening, but sitting in a concrete shelter at night waiting for the sirens to go off can naturally lead to introspection. I had no intention of being a pioneer. It was a simple search for my own personal happiness. I felt like any other choice would be the wrong one. It was too much of a big and scary decision to think at all about how my career would shape out. There were deliberate risks I took and risks that I didn’t expect and needed help with. Back in the states, I was the first officer in my military unit to be out as bisexual after the repeal of “Don’t Ask, Don’t Tell” (DADT). This was a deliberate risk: my personal and professional lives didn’t intersect, but I thought that someone should be out as an example if other soldiers wanted to come out. I still felt like I had a vulnerable secret, as being transgender was still prohibited, and my plans to transition after departure were carefully guarded even as I changed my legal name and started living as female while on leave. My commanding officer was overtly hostile towards the repeal of DADT. After one staff call in which he used “faggoty” as a term of derision, I grew concerned about whether or not I would be able to leave the Army on my own terms. I need not have been concerned, however. Senior members of the staff met with the commander over his comments, and he decided to leave the Army over the pace of change. Like me, but for different reasons, he was voting with his feet. Returning to graduate school, I was the first openly transgender person in my program. I was one of two transgender students to tell my story to the school’s leaders in an ultimately successful bid to change insurance coverage policies for all students. Not all was respect and smooth sailing; in one emotionally crushing case, two teachers complained to my program admin about me using the restrooms that matched my gender identity (it seems strange to type this - much has changed in eight years). My friends calmed me down, and school administrators used the complaint as a teaching moment for the faculty. The only [other] in the room? As my civilian career advanced at Amazon and at Scale, I became more aware of how being transgender would shape my approach to work. No matter the realities of my professional role, I always feel like a bit of an outsider. It has been argued by folks more articulate and demonstrated in studies that diverse perspectives lead to better decisions, and clearer identification of risks. This is a good thing. The downside to feeling like an outsider is always feeling the need to push myself harder, to prove myself better, and to hold myself to an extremely high internal standard - and to always fear slipping. This approach to work has helped me advance quickly, and made me bold, but it is exhausting at times and ultimately rooted in fear. One of the factors in my recent decision to join Scale was that many company leaders identify with the LGBTQ community. I want to help build the future without worrying so much about how I fit in it. Taking a role at a smaller, fast-moving startup with less formal structure has been awesome. There’s a sense of purpose and mission and a rapid flow of ideas that I feel I can really be part of. Transgender people have always existed in one way or another, before society’s own peculiarities and social constructs started calling it a “new thing.” I wrote earlier that I was the first openly transgender applicant at my grad school program. It is highly likely that there were transgender people there before, and it is likely that some of my classmates were transgender and simply hadn’t decided to transition yet. It is important to remember that an individual story cannot represent a community, especially with big variation in opportunity and privilege shaping peoples’ lives. Advice for folks feeling like outsiders There are a few things that I wish I’d known earlier in my career. One piece of advice I got from a friend has stuck with me: “You will always think about yourself a lot more than others think about you.” That feeling of awkwardness, that one thing you did - most people simply won’t notice or remember. Our memories are inherently selective. Learn from your mistakes but choose to remember the good parts. The other thing I wish I’d understood better earlier is to not be afraid to pick a mission with purpose. Early on, I picked assignments that seemed like the best value for the company’s ability to save money or expand programs and to advance my career. I left jobs and teams that I enjoyed to seek out the external trappings of success. Many people do this, but the feeling of validation seemed so necessary when I was uncertain of my place in society. Finding a team and a voice that makes me feel at home and a mission that serves a purpose I believe in has ultimately been more gratifying. Now, what the people around me think matters more than what the world thinks. Scale believes there is tremendous potential for AI technology to drive a revolution in true efficiency and productivity, across industries - this is a change for which I want a front row seat. I am glad I am able to take that seat feeling like I am no longer an outsider.

Scale Operations leader Willow Primack shares her personal experience for International Transgender Day of Visibility

Willow Primack shares her experience for International Transgender Day of Visibility

Trans Day of Visibility: My Experience at Scale

<h1>Project Ideas and Inspiration</h1>
&nbsp;
Scale recently sponsored Treehacks, Stanford University&rsquo;s annual hackathon.
During this fully virtual hackathon, nearly 1300 active hackers and 114 teams
submitted projects for judging. Out of all teams, 21 submitted projects using
Scale&rsquo;s API.
&nbsp;
The winning project for Best ML Hack was
<a href="https://devpost.com/software/fine-tuned-heads" target="_blank" rel="noopener noreferrer">Fine Tuned Heads</a>,
which proposed a new workflow to assist with the implementation of Computer
Vision. Rather than training a single model from scratch, the encoding and
fine-tuning aspects of a task were separated. The Fine Tuned Heads team
created an automatic data collection pipeline using Flask to automatically
execute scripts to grab .m3u8 URLs from Youtube, then used Scale Rapid to
automate uploading and generating batches of ground truth labels for these
images. According to the Fine Tuned Heads team, &ldquo;this approach can be used to
improve the efficiency of inference in ML tasks, and by engineering our
self-supervised model with an efficient architecture and deploying on
efficient hardware, we can reduce the carbon footprint by over 100x.''
&nbsp;
An additional winner of the API Challenge was
<a href="https://devpost.com/software/accecity" target="_blank" rel="noopener noreferrer">AcceCity</a>, which provides
an ML-powered mapping platform that enables cities, urban planners,
neighborhood associations, disability activists, and more to identify key
areas to prioritize investment in. For their project, the AcceCity team first
used the Google Maps API to send images of street views to their backend. They
looked for handicapped parking, sidewalks, disability ramps, and crosswalks
and used computer vision, by custom-fitting a zero shot learning model called
CLIP from OpenAI, to automatically detect those objects from the images. They
then tested the model using labeled data from the Scale Rapid API.
&nbsp;
Other interesting projects include using Scale Rapid to label map data to
promote sustainable living, and classify patient medical image data.
&nbsp;
Beyond the examples from the Hackathon, one can use Scale Rapid for a variety
of ML projects, including to
<a href="https://scale.com/blog/rapid-cyclegan" target="_blank" rel="noopener noreferrer">anime-fy images</a> and to
train a
sentiment analysis model for movie reviews. Use our <a href="https://scale.com/open-datasets" target="_blank" rel="noopener noreferrer">open datasets</a> to draw
further inspiration for your projects.
<h1>&nbsp;</h1>
<h1>How can you use Scale to build a Successful ML Hackathon Project?</h1>
&nbsp;
One of the most challenging problems preventing teams from creating novel ML
projects in a short timespan is the &ldquo;Cold Start&rdquo; problem, which is when an ML
model cannot draw inferences for users or items about which it has not yet
gathered sufficient information. Especially during the compressed timespan of
a hackathon, obtaining high-quality training data for models is nearly
impossible.
&nbsp;
Scale Rapid is the fastest way to obtain production-level quality labels, with
no data minimums. Using Scale Rapid, your hackathon team can experiment
quickly by:
<ol>
<li>Setting up projects in minutes and receiving initial data within hours</li>
<li>Iterating over edge cases and instructions by getting real-time feedback</li>
<li>Scale to production-level pipelines with precision quality control</li>
</ol>
&nbsp;
To get started, sign up for an account with your university email on our
University Program Page to
receive $250 in Rapid Credits, or check out our
<a href="https://colab.research.google.com/drive/1XZjbVtA2oGOdMp0-v0E80ujMB8tbeh7x?usp=sharing" target="_blank" rel="noopener noreferrer">quick start python notebook</a>.
Rapid currently supports the following annotation types:
<ul>
<li>General Image Annotation</li>
<li>2D Semantic Segmentation</li>
<li>Text Collection/Categorization</li>
<li>Document Transcription</li>
<li>Named Entity Recognition</li>
<li>Video Playback Annotation</li>
<li>Lidar Annotation</li>
</ul>
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/2ac2bcbb-1e7d-4268-2c77-bf6d2623c400/public" alt="Annotation Types Supported in Scale Rapid">
&nbsp;
&nbsp;
If you have your own unlabeled data, add it to your project using local file
or csv upload, import it from previous projects, or upload it from the cloud
(S3, GCP, and Azure). Otherwise, check out our
<a href="https://scale.com/open-datasets" target="_blank" rel="noopener noreferrer">open datasets</a>.
&nbsp;
&nbsp;
Once you receive task responses, visualize your data in Scale&rsquo;s dataset
management platform, Scale Nucleus, by navigating to the data tab under your
project and clicking &ldquo;Visualize in Nucleus&rdquo;. Nucleus helps you understand
the strengths and weaknesses in your dataset, analyze the long tail of your
dataset, collaboratively edit labels, and explore metrics, failure cases,
and confusion matrices linked to your training data.
&nbsp;
&nbsp;
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/899fda4e-d4ab-4ed4-7ef4-5fa12e396200/public" alt="Scale Nucleus &lt;&gt; Scale Rapid Integration">
&nbsp;
&nbsp;
Quickly find similar images using the &ldquo;Autotag&rdquo; feature in Nucleus, and
further refine your set of images by providing a few examples of positive
and negative examples. After identifying this subset of your dataset, you
can send it back for labeling through Scale Rapid.
&nbsp;
&nbsp;
Scale AI&rsquo;s products help enable and strengthen a team&rsquo;s ML development cycle
immensely by delivering high quality data for any project. As you or your
team continues to maintain and improve your project post-hackathon, use
Scale AI for consistent, reliable data in order to create, validate, and
maintain production for your high-performing ML models.
&nbsp;

Scale AI sponsored Treehacks: Learn how Scale Rapid powers successful ML hackathons for fast, efficient development.

Using Scale Rapid for Hackathon Projects

How to Use Scale for Your ML Hackathon Project

<h1>Background</h1> At Scale AI, we’re always looking for ways to apply machine learning (ML) in our internal systems to deliver the best results for our customers. We’ve previously written about Scale’s <a href="https://scale.com/blog/how-we-scale-machine-learning" rel="noopener noreferrer" target="_blank">ML-powered pre-labeling</a> as well as our <a href="https://scale.com/blog/autosegment" rel="noopener noreferrer" target="_blank">active tooling, like Autosegment</a>, which helps our Taskers label faster and at higher quality. In this piece, we discuss how we think about and incorporate some basic tenets of design thinking to develop ML systems that make a difference for our labeling pipeline. <a href="http://web.mit.edu/jrankin/www/engin_as_lib_art/Design_thinking.pdf" rel="noopener noreferrer" target="_blank">Design thinking</a> is a mindset and set of practices which foster the collaboration required to solve problems in human-centered ways and was notably popularized by organizations such as IDEO. It’s useful for machine learning engineers who need to understand the full scope of how the system they are building will be used and are responsible for both business and operational outcomes. We’ll describe how design thinking techniques can be applied to scoping machine learning problems and introduce some concepts to help business stakeholders communicate more effectively with ML engineers.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/8a41ed7a-4287-491b-6202-1f80a4c29a00/public" alt="Phases of Developing an ML System">Idealized phases of developing an ML system. In practice, it’s not this linear.In our experience, the different phases of developing an ML system are: <ol><li>Discovery - Locate a business or operational pain point</li><li> </li><li> Design - Map the different ways a new or existing model could fit into the</li><li> extant pipeline</li><li> </li><li> </li><li> Decision &amp; Alignment - Align with operational and engineering</li><li> stakeholders on scope and resourcing</li><li> </li><li> </li><li> Development, Integration, Deployment - Deliver the system. Build data</li><li> pipelines, train models, integrate into the pipeline, etc.</li><li> </li><li> </li><li> Measurement &amp; Experimentation - Identify the effect of the ML system as</li><li> an intervention on the existing pipeline.</li><li> </li><li> </li><li> Continuous improvement - Monitor and retrain to account for distribution</li><li> shifts and new edge cases</li><li> </li><li> </li></ol> This is a descriptive mental model, not a prescriptive blueprint. In reality, a project is always moving between different phases and can often be in multiple phases at the same time. For example, measurement and development form a tight iteration loop. But, it’s still useful to think about these phases in isolation. Concepts from design thinking are mostly employed in the Discovery and Design phases when the problem statements are the most inchoate and ill-formed. In these phases, the need or specific outcome is communicated to an ML engineer and an initial strategy to address that need or deliver that outcome is designed. We focus on the Discovery and Design phases in this blog and cover the remaining phases briefly.<h1>Phase 1: Discovery</h1> An ML system gains life when a need is discovered. By need, we distinguish between a request for a specific feature, such as a linter (a tool we use to ensure data quality), and the underlying operational or business pain point or problem - such as a new labelling capability requiring ML support. It’s not uncommon for operational or business teams to stumble into the <a href="https://en.wikipedia.org/wiki/XY_problem" rel="noopener noreferrer" target="_blank">XY problem</a> by pattern matching and requesting a clone of an existing feature, without realizing that their underlying problem is actually quite different. Or, they might ask for something which is incredibly difficult, unaware that a slightly modified version would save literally months of development time. So an ML engineer should always hunt down the true problem that needs to be solved. Pain points and problems can be project-specific or exist horizontally across business teams or units. Historically, many of the problems that the ML team at Scale have tackled are project-specific and brought to us by our cross-functional stakeholders. For example, a project manager running into issues with a labeling project or a product manager looking to launch a new feature will surface these project-specific issues to our team. Tackling these isolated problems can result in marooned solutions which aren’t readily reusable. Solving horizontal issues, in comparison, maximizes our leverage, since the ML systems we build will then scale with our business. However, cross-business issues that can be solved with ML are often harder to identify because of the sheer operational complexity. These issues require a more active discovery process and take more time and deeper alignment with partners. The problems with the highest stakes are not obvious to ML engineers, who don’t get day to day exposure to them, even as these frictions impose significant pain on the operations or business side. To get a sense for this, imagine explaining the pain of managing multiple python environments and package versions to an operational lead - it can be incredibly painful for you, yet esoteric, even cryptic, to them. As another example, while working on an image semantic segmentation project, we learned that pre-labels generated by a model specific to that project were sometimes ignored by taskers, but the root causes were unclear. A close investigation identified a number of culprits, ranging from the inflexibility of pre-labels to having optimized for the wrong metric during development. <a href="https://scale.com/blog/autosegment" rel="noopener noreferrer" target="_blank">Autosegment</a>, developed by Sean Li, Saleh Hamadeh, and the two of us, aims to tackle these problems. Autosegment solves a challenge for all image segmentation projects and isn’t restricted to a particular project. Learning about this challenge and building the appropriate solution required input from numerous stakeholders - from the project leads, who wanted to accelerate taskers; the taskers themselves, who used our tools and knew what worked and what didn’t; to engineers, who were familiar with the implementation of image segmentation in our interface and understood the kinds of tradeoffs that different tools would be need to make.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/7866cd69-bd81-40f7-4e44-6094a84f2c00/public" alt="Stakeholder concerns">Example overlap of concerns between different stakeholders for a new ML-assisted tool. Identifying the pain points of the end user always requires talking to the end user. Instead of theorizing what could be a problem for them, it’s always more effective to speak to users directly and observe them as they encounter the problem. When building systems to accelerate labeling operations, ML engineers need to build user empathy. A simple way to build user empathy is to spend time tasking and dogfooding our own platform. There is no substitute for first hand experience for understanding what makes tasking hard.<h1>Phase II: Design</h1> After Discovery, the project enters a design phase where an initial approach is sketched out - what kind of model to build, what sort of data is needed, the timelines, and so on. Designing effective ML systems at Scale requires two things: a clear need, which we’ve discussed above, and knowing the many different ways a model can fit into the labelling pipeline. This last point is worth further elaboration. The same machine learning model can be used for a variety of operational purposes. For example, a model that predicts bounding boxes for cars can be used to:<ol><li>Estimate the number of boxes in a task (task difficulty estimation)</li><li>Prepopulate the scene with bounding boxes (pre-labelling)</li><li> </li><li> Suggest fixes to bounding boxes made in real-time by a tasker (active</li><li> tooling)</li><li> </li><li> </li><li> Identify missing boxes or unexpected boxes in a completed scene (linting)</li><li> For each of these, the inference API could have the exact same request and</li><li> return formats. However, each of the four approaches above require different</li><li> operational adjustments, not all of which may be feasible. Task difficulty</li><li> estimation makes assumptions about what makes a task difficult or easy for</li><li> humans and implicitly maps these to legible subpopulations of taskers.</li><li> Active tooling can require nontrivial operational investment in teaching</li><li> taskers how to use the new tool and engineering lift to build new frontend</li><li> components. Building a useful ML system means designing how an ML model will</li><li> interface with the rest of the operational and engineering systems, in</li><li> conjunction with actually training the ML model.</li><li> </li><li> </li></ol><img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/8e6d0b0b-5185-4a7c-227e-03350b1bdd00/public" alt="One model, many uses">The same model may be applied in different ways to an existing pipeline, requiring different accommodations by engineering or operational teams. We’ve noticed two particular traps for ML engineers which can lead them on wild goose chases during the design phase. The first is misalignment between the machine learning problem and the business problem. The second is confusing the customer and the end user of a system. ML engineers are trained to use models to solve machine learning tasks, like box detection or text classification. For a box detection dataset, you’d use a box detection model. For a text classification dataset, a text classification model. When the business problem looks like an image classification problem, you might reach for an image classification model, only to discover that what you really need to solve is tabular regression. Faced with using ML to assist, say, labelling semantic segmentation training data, a common reflex is to train a semantic segmentation model to do the labelling. But if you already had a capable model, you wouldn’t need to collect training data for it! The mistake here is to apply the familiar task paradigm - semantic segmentation - to solve the business problem - accelerating semantic segmentation. Many operational questions take the form “Is something X or not X?” To an ML engineer, this sounds like a classification problem, and it’s easy to reach for a classification model which directly answers the question. Putting aside whether modeling X directly is feasible, it’s often the case that we already have an operational or engineering system which could answer this question more effectively with some additional information. In this situation, it may be faster to figure out how to extract the augmenting information rather than replicate the existing system altogether; a strong reason to apply a modeling scheme that looks very different from the modeling question.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/3505a43e-5c45-4274-cdd3-2e27242e0600/public" alt="Leveraging existing systems">Modeling the business problem to solve doesn’t always mean reaching for the most obvious machine learning task paradigm. The second trap is mistaking the end user of a system with the people who request or manage the system. When we build a tool like <a href="https://scale.com/blog/autosegment" rel="noopener noreferrer" target="_blank">Autosegment</a>, even though the end user of the active tooling is the Tasker, the internal customer is actually the Delivery Operations team. What we build has to meet the customer’s expectations, so even if Taskers enjoy using our tool, we need to make sure to move the needle for the Delivery Operations team. But the expectations and needs of the end user may differ or conflict from the internal customer (e.g. latency tradeoff with cost). In order to avoid falling into these two traps, ML engineers have to be deliberate in how they transport the business problem into a machine learning context; and always keep in mind the different stakeholders.<h1>Phases III - VI: Decision and Alignment Through Continuous Improvement</h1> After the initial Discovery and Design phases, the project needs a go / no-go decision by both the ML team as well as cross-functional teams. In this phase, all teams must decide whether to allocate the necessary resources to the project. If the ML team decides to dedicate resources but the cross-functional team cannot commit the time or spare the bandwidth, the effort will ultimately fall short. Some factors teams must consider when making this decision include: prioritization, level of effort needed, and available bandwidth. Once all teams commit to the project, we can develop the ML-specific components of the ML system. This is when we build data pipelines, train models, and ship an endpoint. Once the model is developed, the project nearly always needs engineering and operational integration. For example, if we are developing a new pre-labeling model, the model requires integration with our labeling pipelines to attach the pre-label to the task. Now the project is actually deployed. Deployment should also follow best practices: meaning the feature should only be enabled for a small percent of tasks or users at the beginning, making it easier to revert if there are any issues. The system should then be constantly measured. Especially in the first few weeks, close attention should be paid to ensure no issues arise. We then use the metrics and qualitative feedback gathered in the measurement step to continuously improve the model. Models often need to be retrained to account for new edge cases. As we maintain the production model, we often encounter data distribution shifts or taxonomic changes which must be addressed.<h1>Conclusion:</h1> We hope this post provided some insight on how we design our ML systems and how you and your teams can effectively partner with horizontal teams to deploy models that serve your business needs. Throughout the phases of ML development, we strive to ensure alignment with our cross-functional stakeholders. While gaining and maintaining alignment takes time and deliberate effort, it allows us to unlock capabilities on our platform that wouldn’t otherwise be possible.

Understanding operational and business context is critical to shipping ML systems that work. 

How we leverage concepts from design thinking to design ML systems in our internal processes to deliver results for customers.

Design Thinking for ML Systems at Scale

As a member of the U.S. Air Force Rescue and #AFWERX, I have been working to modernize the tools used by Search and Rescue to increase the odds of survival of those lost at sea. Every year, hundreds of individuals find themselves isolated and at the mercy of the ocean, creating a dangerous scenario which is impossible to navigate without the best technological tools and deep expertise. Being in the ocean is one of the more extreme environments to be isolated. The conditions that make it hard to survive also make it difficult to locate and rescue the isolated; rescue is rarely a simple mission. Attempting to modernize the Search and Rescue model is arguably just as difficult. Systems are out of date: the “how” hasn’t changed in over 20 years. And changes that have been adopted, have not been implemented at the speed of relevancy. Five years ago, I began exploring different approaches to solving Search and Rescue’s toughest problem statements. During this discovery process, two significant experiences led me to become a Fellow at Scale. In my first experience, I was deployed as a United States Special Operations Command (SOCOM) Tech Advisor as a Personnel Recovery Subject Matter Expert (SME). I had the opportunity to share these problem statements with Josh Wolfe, co-founder of Lux Capital, and the rest of the SOCOM Tech Advisory Panel team. Josh shared his opinion on how the DoD could use a page out of the venture capital playbook: to move faster with sustainable speed in developing solutions. Seeing how Josh was investing in startups that invent the future inspired me. I began to engage with early-stage startups to find new solutions that would save lives. The second instance was at a Hacking for Defense session at Stanford University. This time, I was working with young college students learning the lean startup methodology to solve DoD problems. Steve Newell, the founder and CEO of BMNT, introduced the Search and Rescue problem statement to the class. He presented it in such an elegant way—I could physically see his inspirational words produce passion on the young data scientists’ faces. At this moment, I realized that AI developed to save lives would be the catalyst for large-scale DoD adoption. A few years later, and several AFVentures Small business Innovation Research contracts later, Forrest Underwood introduced me to Shift.org. This was the opportunity I couldn't pass up: the chance to work inside a startup for 8 weeks. Shift was able to recognize a critical flaw in my strategy to solve problems. How can you get the DoD to adopt industry AI solutions without possessing a well-developed ML pipeline? For this reason, I think Shift matched me with Scale AI. Scale AI is the best-in-class at providing the entire ML lifestyle to its customers so they can focus on the real, pressing problems at hand. This is exactly what's needed when a DoD customer asks for an AI solution. It is a misunderstanding of the requirements surrounding ML that can be difficult when attempting field AI solutions. Without a mature ML pipeline, it’s very difficult to have your model perform correctly and continue to improve as the extremely dynamic world evolves. On the first day on the job, I attended #TransformX, a virtual conference with the CEO Alex Wang, a community of leaders, visionaries, practitioners, and researchers across industries. This was probably one of the best onboarding or familiarization phases that I've experienced in 20 years. Spending 2 days listening in on thought leaders discussing the art of what's possible in a long forum was extremely insightful. Throughout the entire fellowship experience, I compared how Scale operates against Air Force Institutions of which I’ve been a part. One observation that stood out to me was the holistically flat organizational rank structure of Scale. After seeing the level of engagement that Alex Wang had during TransformX and the scope of his responsibilities, it was easy to draw parallels between Alex and a Geographic Combatant Commander. One welcoming attribute of Scale was the inclusion of ambitious ideas. It is understood that the company is at the edge of the AI frontier, and new ideas are required to move forward quickly. This is what has allowed Scale to grow at an incredible rate; it’s also the attribute that the Air Force is trying to re-install into its ranks. Inside Scale, I had the opportunity to work for the Federal division. The level of talent and prior experience of the team was truly awe-inspiring. I had a false assumption that I would be working with individuals without experience in the Armed Forces. I was surprised when I learned that there was a rich mix of individuals from both the DoD and Silicon Valley. At first, it felt like I walked into a Special Operations Group with Senior Instructor Pilots, Chief of Intel, and the Ground Forces Commander discussing an Operations Order. Except, everyone was in business casual and there were lead engineers in the conversation. For Scale to recognize the talent pool coming out of the DoD was exciting to see. The method of mixing operators, strategists, and engineers inside is not common practice for the Air Force. The standard Air Force approach is to silo these different functions on a day-to-day basis. Even my two cents were considered as the new guy. That says a lot about an organization. The art of creating what's possible is something that Scale was always discussing with potential customers. This wasn't a "fake it until you make it” model. The conversation was followed up with things that were deployed in that space to demonstrate their success. My gained understanding of what's possible is what I will take back to the Air Force. Thank you Scale.

c63a1be0661f1594644b740c08bede0e.jpeg

Bon Strout details his experience as a Scale Fellow

Scale Fellow Spotlight on Bon Strout

Shift Ventures Fellow Spotlight: Bon Strout

<h2>Update:</h2>
 We have received an overwhelming response to our announcement of free, AI-ready datasets in support of Ukraine. Nearly a hundred AI companies, researchers, developers, and GIS practitioners have offered to support the initiative with Scale &ndash; an inspiring global community of AI Developers for good.
Given the high demand, we will provide the datasets in phases starting with requestors with the greatest need, followed by a broader community. At this time, our focus is on providing Structure Damage Assessments in Kyiv, Kharkiv, and Dnipro.
The path forward is now two-pronged &ndash; our top priority is the delivery of damage assessments to those with urgent need.
The second prong of our effort is to provide AI-ready training datasets for Damage Assessment to the broader AI community, starting with a structure recognition training dataset.
Today, we are sharing an initial dataset with the broader AI community: <a href="https://drive.google.com/uc?id=17lpMFngiqeegIm1U0RzUYfB-9GjmI7fk&amp;export=download" target="_blank" rel="noopener noreferrer">link to download dataset</a>.
Note: the corresponding ZIP file's SHA256 Checksum is: 379c394dab6a4d7f1358625d086ebea4b39aaa124548e8c079bee16fb574bc53.
We appreciate your patience as we assess and prioritize requests for data and data development. We look forward to continuing to support Ukraine alongside other AI Developers leveraging critically important open-source and commercially available data for this important work.
Sincerely,
Scale AI Team
&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;&mdash;
At Scale, we believe in using AI technology in support of democratic values. That&rsquo;s why starting today, Scale will be providing a series of AI-Ready datasets that algorithm developers can use to rapidly train and deploy AI in support of Ukrainian operations. By providing these datasets at no cost to national security practitioners, we hope to support a diplomatic solution and swift end to this conflict.
Diplomacy is fueled by insights; foundational to that is quality, trusted data. We have recently created strategic partnerships with commercial satellite imagery providers in order to develop these datasets, which include Aircraft Detections, Structure Damage Assessments, Change Detection, and the JSON and GeoJSON annotations. We encourage AI Developers to contact detections@scale.com to receive this data free of charge.
To better understand how these datasets can be used, Scale presents a contemporary use case of our partnerships with national security leaders. We deliver an AI-powered change detection solution capable of identifying and tracking military formations across the globe.
The video below demonstrates continuous monitoring and improvements by Scale&rsquo;s ML engineers and deployment strategists. We provide timely and accurate insights through our AI Data Readiness platform, our Test and Evaluation platform and our continuous ML Ops tooling.
Leveraging Scale's AI/ML ops platform, we continuously develop and hone deep learning models to describe scenes and detect objects across a range of inputs from commercial satellite imagery providers.
<iframe class="ql-video" style="width: 564px; height: 282px;" src="https://scaleai.wistia.com/embed/medias/hphfibwbn6" width="100%" height="400" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
We are now delivering insights to U.S. national security customers from this full-stack, multi-intelligence, human-teaming product, including on Russian formations across Europe and Eurasia.
Here, on Russian air and ground order of battle (OOB) data, we successively identified OOB changes across the EUCOM area of responsibility (AOR) over the past several months. We used high-resolution synthetic aperture RADAR(SAR) imagery from Capella Space and high-resolution optical imagery to deliver high-accuracy, time-sensitive insights.
&ldquo;In these intricate and time-sensitive scenarios, it is critical that we deliver insights from multiple, disparate datasets at the speed of relevance,&rdquo; said Mark Valentine, Head of Federal at Scale. &ldquo;We are proud to demonstrate the power of our platform, and our approach to human-machine teaming to support national security leaders and practitioners.&rdquo;
Monitoring the Russian OOB on the border of Ukraine using commercial satellite imagery, and other non-traditional Intelligence, Surveillance and Reconnaissance (ISR) sources requires adaptation to a variety of challenges in a constantly-evolving field.
The challenges in using AI to analyze OOB changes include: exigent scenarios presented by weather, disparate image varieties and quality, and the inclusion of relatively rare military objects such as transporter erector launchers (TELs), for which there are limited real-world observations.
The depth of ontology can also present complications: the presence of multi-generational aircraft, ground vehicles, RADAR and TELs mean that models need to be trained against tens of thousands of observations in a range of environmental conditions.
If you are an AI Developer who would like to receive this data free of charge, please contact <a href="mailto:detections@scale.com">detections@scale.com</a>.

ae82457827ce45793eefc6460623a063.jpeg

Scale AI provides free datasets for national security partners

Scale AI providing free datasets to help national security partners gain current insights into Russia-Ukraine conflict

Scale Rapid is a quick and easy way to get clean, high-quality labeled data for your next ML project. We used it to train a <a href="https://scale.com/blog/imdb-rapid" rel="noopener noreferrer" target="_blank">sentiment analysis model</a> in a previous post - and we’re back now to show you how to use Scale Rapid to train an unsupervised model! In this blog, we are going to use <a href="https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix" rel="noopener noreferrer" target="_blank">CycleGAN</a> (<a href="https://arxiv.org/pdf/1703.10593.pdf" rel="noopener noreferrer" target="_blank">paper</a>) to anime-fy images with Scale Rapid. CycleGAN is trained in an unsupervised manner, meaning each X input does not need to correspond 1:1 to a y output in order to train the net. Instead of using Scale Rapid to label training data, we used it to clean data. We separated a group of images scraped from the internet into categories. This classification allows us to make the two inputs necessary for CycleGAN - a class A (real-life images) and class B (Japanese anime-style images). To start, we downloaded 10k images from Google images from a variety of different searches. At a glance, the corpus had all sorts of images. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/27b06dda-1661-4805-c721-0ec70ee20300/public" alt="Sample images from Google">Class A contains real-life images and Class B contains anime-style images. Image 3 belongs to Class A, and Image 2 to Class B. Images 1 and 4 do not belong in the dataset as they are neither real-life nor anime-style images. The ask for Scale Rapid is to discriminate between these three types of images: a real-world, realistically edited photograph, a Japanese-style anime scene, or neither. Rapid responses are then used as ground truth to establish which images should and shouldn’t be in the dataset we train CycleGAN on. Rapid’s categorization endpoint supports image categorization - and it's only 8 cents per image! Because CycleGAN is trained on unsupervised data, we don’t need to pair class A and class B images - we can simply separate them from one another and train. <h2>Using Scale Rapid</h2> Scale Rapid works in two phases- first, the user sets up the project &amp; instructions, and runs small calibration batches to ensure the initial results are as expected. Once quality on those looks good, larger production batches may be uploaded for labeling. For this application, we set up a categorization project. The taxonomy is just one multiple-choice question: What type of image is this? <ul><li> </li><li>Anime (japanese animation) image</li><li> </li><li> </li><li>Photorealistic (real-life) image</li><li> </li><li> </li><li>Neither</li><li> </li></ul> Scale Rapid also has a feature called long hints, which allows us to prompt labelers with specific things to think about upon choice selection via a tooltip. For the first choice (Anime (japanese animation) image) I clarified with the following long hint: "Japanese animation refers specifically to anime, as in the instructions. Other cartoon-style or generated images belong in the neither category." We begin by uploading a calibration batch, and follow up with production batches. The labeling results are easily parseable via JSON. <h2>Scale Rapid Results:</h2> A random audit of tasks found just one mistake out of 200 randomly selected labeled images. A few examples of correctly labeled images are below, organized by class.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/d6e522b3-b00e-4e91-47f5-3f709d144300/public" alt="Examples of correctly labeled images by class">With the help of Scale Rapid, raw data from the web was transformed into cleaned and confirmed data and separated into class A, class B, and neither. <h2>Model</h2> CycleGAN produces mappings from class A to class B and vice versa using two different types of losses- adversarial loss and cycle consistency loss. Adversarial loss incentivizes the mapping to generate images that look similar to the target set. We achieve this by optimizing against an adversary D–a discriminator that tries to separate the generated images for the target class from actual data from the target class. Cycle consistency loss incentivizes the mapping to produce images that look like they fit the distribution of the target class.This occurs by minimizing the difference between original data and that same data transformed into the same target class.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/cdcd9177-a511-4863-70fa-6a9c1194dd00/public" alt="Cycle consistency loss">Cycle consistency loss is calculated by the difference between the untransformed class A image and the double-transformed class A image. The mapping aims to minimize this.We trained CycleGAN with our Scale Rapid sorted dataset.<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/3c9b78d0-1aaa-4d49-ad1d-e7d91e37ca00/public" alt="Loss Function Graph"><h2>Model Results:</h2> We are now able to anime-fy images! A few examples:<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/5a438327-e4b7-41e5-a19b-2cd457c94d00/public" alt="CycleGAN Images"><a href="https://scale.com/rapid" rel="noopener noreferrer" target="_blank">Try Scale Rapid</a> for your own project today!

In this blog, we explore combining Scale Rapid Labeling with CycleGAN to anime-fy images.

Using Scale Rapid to train an unsupervised model.

Using CycleGAN to Train Unsupervised ML Models

Using Rapid and Nucleus together will get you better results than using Rapid alone.

Use Scale Nucleus with Rapid to Uncover Errors Smartly

Peer-Reviewed Doesn’t Mean Perfect Data

Mission critical industries including logistics, financial services, government, and healthcare rely on high quality document data extraction in order to optimize business workflows, save money, and improve customer experience. In this article, we explain the conventional approaches for document processing, deep dive into drawbacks with these solutions, and introduce a new approach that enables higher accuracy, even across the most challenging document types. <h3>What is OCR?</h3> Optical character recognition (OCR) is a class of document processing technologies that, as the name suggests, reads characters from text on a page. Originating in the 1920s, OCR has since evolved to support use cases involving data extraction from digitized documents. There are two kinds of OCR technologies: First, generic OCR, which pulls all of the text on a digitized document (image, PDF, etc.) and outputs a raw list of the words on a page. With a number of improvements in character recognition technology, generic OCR has become suitable for distinguishing words and reading complex font types. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/98229f9a-1e6c-4a80-7416-5a38dfbd1d00/public" alt="Generic OCR: extraction/digitization but no field classification"> Second, intelligent OCR, which has become a popular alternative since these systems can automate data extraction for key fields of interest rather than just receiving a blob of all the text on a page. Intelligent OCR is built on generic OCR, but enables a user to define a specific template for a document, manually specify all the fields they need extracted, and automatically get only the structured data they need for all similar-looking documents. <h3>Problems with OCR</h3> There are clear challenges with both of these conventional OCR approaches that reduce the quality of data extraction. Generic OCR simply outputs all the text on a page and doesn’t extract only the key information you might need for downstream workflows. These solutions are typically paired with complex regex or hard-coded heuristics that could require a team of nearly 4-5 engineers just to support parsing out required fields and maintaining the infrastructure. Even then, the variability in text layout and document format ultimately results in poor data extraction accuracy (~50-70% for simple documents, 10%-60% for complex unstructured documents). Intelligent OCR can extract fields you need in a document, but requires manually creating templates and rules to define the location of those fields for every document layout. This approach can quickly become painstaking and incredibly difficult to scale as organizations expand use cases. These solutions also fail when it doesn’t find a matching template on new layouts or when you need to add new fields to extract—common in real-world scenarios that have high volumes or complex, unstructured document types. While intelligent OCR approaches might get up to 70-80% data extraction accuracy with heuristics, it still requires significant engineering and manual operations support to ensure errors don’t fall through the cracks. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/6eb13fad-3577-422f-8cc9-51fc0a1eb100/public" alt="Intelligent OCR: extraction and digitization, classification from templates only"> <h3>To summarize:</h3> <ul><li> </li><li>Generic OCR simply extracts text but doesn’t identify the specific fields you need extracted. This approach requires significant engineering effort to maintain the infrastructure and yields low accuracy data.</li><li> </li><li> </li><li>Intelligent OCR can extract your desired fields but requires manually defining templates and hard-coded rules. This approach can get to higher accuracy levels than generic OCR but still results in many errors and doesn’t scale to large volumes and high variability use cases. </li><li> </li></ul><h3>Why use Adaptive ML Document Processing instead?</h3> Running OCR is the first of many steps needed to have a robust document processing solution that can extract data at the highest quality. While these solutions may get you part of the way there, they don’t deliver a complete solution because OCR still requires significant engineering and operations support in order to pull out key fields and fix errors. While we hear about machine learning frequently in moonshot applications like autonomous driving and Artificial General Intelligence, the technology can be significantly useful today in enterprise use cases. Leveraging machine learning is key to getting a high quality and truly automated document processing solution even across highly variable and challenging document types. Rather than relying on off-the-shelf methods or ones that need manually defined templates, an end-to-end machine learning based approach can enable a robust solution with higher accuracy results for highly variable unstructured and structured documents. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/8d041be1-a4c2-4e5b-7641-785b4c115000/public" alt="Scale Document AI Adaptive Document Processing: the Best of All Worlds"> However, setting up these systems in-house requires significant investment across the machine learning stack in order to deploy a production system, including data labeling, model training, and continuous monitoring. Choosing the right partner to do this at scale can help you get higher accuracy, lower latency, and save time and money to spend on your company’s core competencies. <h3>Why Scale?</h3> Scale Document AI supports customers with quickly deploying a complete document processing solution with guaranteed quality SLAs for any of their document types. We use our own in-house OCR engine for identifying the text on a page, but don’t stop there. What’s unique about our approach is that we leverage base models built on state of the art Computer Vision and Natural Language Processing research. These models are trained on millions of data points and we further refine them for each customer use case. This fine-tuning uses Scale’s proprietary data labeling and ML training infrastructure that we’ve developed over the past few years to deliver a solution that consistently performs at high accuracy – even across unique edge cases, customer requirements, or variable document types. Scale provides Adaptive ML document processing for many enterprises and organizations across financial services, logistics, manufacturing, healthcare, real estate and government. For example, Scale delivers fast extraction from messy invoices <a href="http://scale.com/customers/brex" rel="noopener noreferrer" target="_blank">for Brex</a>, accurate and timely data from complicated shipping paperwork <a href="http://scale.com/customers/flexport" rel="noopener noreferrer" target="_blank">for Flexport</a>, and essential data out of titles, mortgage applications, and deeds <a href="https://scale.com/customers/states-title" rel="noopener noreferrer" target="_blank">for Doma</a> in real estate transaction processing. If you’re looking for a solution that’s higher quality, lower latency, and free of the hassle of creating and maintaining templates, <a href="https://scale.com/document-ai" rel="noopener noreferrer" target="_blank">learn more and contact us</a>.

OCR and even Intelligent Document Processing fall short, while AI-based extraction delivers.

With adaptive ML models performing extraction, Scale Document AI goes far beyond OCR.

Comparing Scale Document AI to Traditional OCR

URL redirections for Website. Disclaimer: takes 5~ min for each added or edited.

url-redirects

Use relative path. use `/blog/foo` not `https://scale.com/blog/foo`

blog-filter-type

homepage-quotes

logos

homepage-content

quotes

blog-category-model

announcement-banner

homepage-resources

homepage-event

custom-fonts

author

Event model for new marketing events page

event

Optional second image intended to show on the grid view, should be a more squared image closer to 1:1 aspect ratio than the one used in the Cover field.

guides

announcement-news

homepage-research

legal

slugs must be all lowercase. ie: ‘my-slug’

eval

datastream

customers

Display order on the page /customers. Higher values show first

summit-2023-speakers

event-model

Toggle if the event is featured. Featured events will be moved to the top of the list

basecamp-agenda

basecamp-speakers

Example: https://www.twitter.com/username

Example: https://www.linekin.com/in/username

summit-2023-agenda

summit-2023-partners

enterprise-use-cases-categories

enterprise-use-cases-categories-items

partners

data-samples

enterprise-use-cases

Use only one image or video option. Add another item to add another source.

research

blog-model

Option to list or not list this post in the Blog index. Enabled = not list in the index, Disabled = list in the index

Products

Enterprise

Government

Resources

Customers

Leaderboards →

Blog