Prathik Naidu

Document AI is powered by machine learning (ML) models that process documents at high accuracy and speed. When these models are fine-tuned for a customer, they become even more accurate and regularly achieve near-perfect accuracy levels and do not require any human intervention. Scale engineers continuously work to improve these models by creating new tools to make sure Document AI is highly reliable for the most critical industries we serve. This blog will highlight one of those tools, a technique called “linting”, and how it helps make Document AI’s models more accurate.<h2>What is Linting?</h2>At Scale, linting is the practice of creating rules or developing machine learning models that automatically detect errors in training data used to create models. In the case of Document AI, we leverage two types of linters: simple, quick linters and more complex ML-based linters.&nbsp;Here is an example of a simple linter comparing field types and their responses within a document.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fb8b843651dad4ba48f5831c07b48c6f2" width="755">In the example above, the letter ‘O’ should not be in a numerical, monetary amount field and our field-type linter would flag this example for correction.Another example of the simple quick linters is a linter that enforces known business rules. For example, a linter can ensure an invoice subtotal is equal to the sum of all line item totals.<h3><img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F9e0a34e10d464195911c07b617ee27c7" width="574"></h3>In the example above, the subtotal on the right ($16) is inaccurate and our linter would flag this example for correction.These simple linters are helpful where straightforward rules can easily be applied. Simple linters, while useful, can be implemented by most. What’s more unique about Scale Document AI is how we go beyond the mainstream methods and tools in the document industry to achieve high-quality.&nbsp;Scenarios that require a higher level of capability or understanding are better suited to machine learning (ML)-powered linters instead of rule-based linters, which is Scale’s specialty. Our specialized ML linters are trained to perform specific tasks, such as learning to compare differences in a transcribed text and the visual representation of the text as it appears on the document’s page. In this example, when the linter finds a meaningful discrepancy, it automatically corrects it or flags the document for review by highly experienced and skilled reviewers. This human-in-the-loop review is optionally available based on customers’ needs and preferences.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F626b6d1309fa4dc28c28d6d75279abb6">In the example shown above, we trained an ML-based linter to understand the context to know that “Account #” on the bottom left refers to a bank account number as it is in a section with ACH instructions while “Account Number” on the top right is referring to a business account number. If “Account Number” in the example on the top right had been identified as a bank account number, the linter would flag this for correction.ML linters are inherently more difficult to implement than rule-based linters as they have the same challenges as developing any other machine learning models. Developing an ML model is a statistical process prone to variation. That variation can manifest as unreliability or as overfitting. For example, unreliability can lead to an ML model producing wildly different predictions on two slightly different documents. With overfitting, the model learns the detail and noise in the training data and fails to perform on new data. In the case of linting, the ML models are trained on data with errors to help identify and flag other errors. The data used for training linters is inherently not of the highest quality; therefore it’s common that while the ML linter can still make decent suggestions, the model itself is not the most performant. This requires our ML team to place more restrictions and guardrails, like ensuring the confidence threshold is set high to only flag errors and make suggestions on things the model is fairly certain about. This prevents the model from producing too many unreliable linter suggestions that lead to human reviewers simply ignoring or not trusting these suggestions.&nbsp;Developing and deploying ML linters is also an iterative process with a feedback loop. As we deploy ML linters for our document processing models, the document processing models improve upon being retrained with higher quality data. As the document processing models improve, it gives better first results as well as learning other errors to surface to the human reviewers.<h2>How Do Linters Improve Models?</h2>The power of machine learning to recognize patterns between input and output data is its greatest strength as well as its Achilles’ heel. If the training data contains errors, the model will learn the wrong task. Having the right linters in place increases the accuracy of machine learning models by ensuring the data used for training is of the highest quality.&nbsp;For Document AI, we began by designing simple linters to check field types such as monetary amounts, quantities, and ZIP codes. We then expanded beyond these simple linters and implement ML-based linters to check for complex fields such as addresses with varying formats that we would need to parse and enforce.&nbsp;To make things more concrete, let’s think about the use case of invoices. At first glance, it seems straightforward to recognize an invoice: they all list the products sold or the services rendered, the price of each, and how payment should be made. However, invoice layouts, details, and content vary widely from business to business. Some invoices are particularly formal, containing everything from company bank account numbers and line item descriptions to account manager names and contact information. On the other end of the spectrum, an informal email between longtime business partners can also serve as an invoice. The unstructured and ever-changing nature of invoices make it challenging to model comprehensively and accurately.When building Document AI’s machine learning model for processing invoices, using linters resulted in a 16.3% relative decrease in transcription errors in the training data. This relative decrease in transcription errors is crucial, as the misspelled text makes it impossible for our document parsing models to extract the right information. This linter also helped remove the need for an additional layer of human quality assurance (QA) review while still ensuring that training data is pristine.&nbsp;Training our models on the higher quality training data led to a 14.3% relative increase in our document processing model’s accuracy. This improvement translates to increased model generalizability and robustness, and more accurate models mean fewer manual reviews, providing additional value for our customers.&nbsp;Linters make Document AI more accurate and hence deliver higher quality models to our customers who rely on high accuracy document processing.<h2>Parting Thoughts</h2>Scale Document AI distills insights from your messy, changing document data and delivers fast, accurate models tailored to your business without the need for templates. If you’re interested in learning more, take a look at our <a href="https://www.youtube.com/watch?v=jWCwVcQCA10" rel="noopener noreferrer" target="_blank">product video</a> or visit our <a href="https://scale.com/document-ai" rel="noopener noreferrer" target="_blank">product page</a>.Acknowledgments:We thank Malcolm Greaves from Scale’s Document AI Machine Learning team for his experiments that laid the groundwork for this post and for frequent consultations throughout the writing of this blog.

Document AI's linters improve model quality by ensuring common and hard-to-spot errors never make it into the training data.

Scale Document AI leverages ML Linting to Improve Document Processing

How ML Linting Meaningfully Improves Adaptive Document Processing at Scale

Mission critical industries including logistics, financial services, government, and healthcare rely on high quality document data extraction in order to optimize business workflows, save money, and improve customer experience. In this article, we explain the conventional approaches for document processing, deep dive into drawbacks with these solutions, and introduce a new approach that enables higher accuracy, even across the most challenging document types. <h3>What is OCR?</h3> Optical character recognition (OCR) is a class of document processing technologies that, as the name suggests, reads characters from text on a page. Originating in the 1920s, OCR has since evolved to support use cases involving data extraction from digitized documents. There are two kinds of OCR technologies: First, generic OCR, which pulls all of the text on a digitized document (image, PDF, etc.) and outputs a raw list of the words on a page. With a number of improvements in character recognition technology, generic OCR has become suitable for distinguishing words and reading complex font types. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/98229f9a-1e6c-4a80-7416-5a38dfbd1d00/public" alt="Generic OCR: extraction/digitization but no field classification"> Second, intelligent OCR, which has become a popular alternative since these systems can automate data extraction for key fields of interest rather than just receiving a blob of all the text on a page. Intelligent OCR is built on generic OCR, but enables a user to define a specific template for a document, manually specify all the fields they need extracted, and automatically get only the structured data they need for all similar-looking documents. <h3>Problems with OCR</h3> There are clear challenges with both of these conventional OCR approaches that reduce the quality of data extraction. Generic OCR simply outputs all the text on a page and doesn’t extract only the key information you might need for downstream workflows. These solutions are typically paired with complex regex or hard-coded heuristics that could require a team of nearly 4-5 engineers just to support parsing out required fields and maintaining the infrastructure. Even then, the variability in text layout and document format ultimately results in poor data extraction accuracy (~50-70% for simple documents, 10%-60% for complex unstructured documents). Intelligent OCR can extract fields you need in a document, but requires manually creating templates and rules to define the location of those fields for every document layout. This approach can quickly become painstaking and incredibly difficult to scale as organizations expand use cases. These solutions also fail when it doesn’t find a matching template on new layouts or when you need to add new fields to extract—common in real-world scenarios that have high volumes or complex, unstructured document types. While intelligent OCR approaches might get up to 70-80% data extraction accuracy with heuristics, it still requires significant engineering and manual operations support to ensure errors don’t fall through the cracks. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/6eb13fad-3577-422f-8cc9-51fc0a1eb100/public" alt="Intelligent OCR: extraction and digitization, classification from templates only"> <h3>To summarize:</h3> <ul><li> </li><li>Generic OCR simply extracts text but doesn’t identify the specific fields you need extracted. This approach requires significant engineering effort to maintain the infrastructure and yields low accuracy data.</li><li> </li><li> </li><li>Intelligent OCR can extract your desired fields but requires manually defining templates and hard-coded rules. This approach can get to higher accuracy levels than generic OCR but still results in many errors and doesn’t scale to large volumes and high variability use cases. </li><li> </li></ul><h3>Why use Adaptive ML Document Processing instead?</h3> Running OCR is the first of many steps needed to have a robust document processing solution that can extract data at the highest quality. While these solutions may get you part of the way there, they don’t deliver a complete solution because OCR still requires significant engineering and operations support in order to pull out key fields and fix errors. While we hear about machine learning frequently in moonshot applications like autonomous driving and Artificial General Intelligence, the technology can be significantly useful today in enterprise use cases. Leveraging machine learning is key to getting a high quality and truly automated document processing solution even across highly variable and challenging document types. Rather than relying on off-the-shelf methods or ones that need manually defined templates, an end-to-end machine learning based approach can enable a robust solution with higher accuracy results for highly variable unstructured and structured documents. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/8d041be1-a4c2-4e5b-7641-785b4c115000/public" alt="Scale Document AI Adaptive Document Processing: the Best of All Worlds"> However, setting up these systems in-house requires significant investment across the machine learning stack in order to deploy a production system, including data labeling, model training, and continuous monitoring. Choosing the right partner to do this at scale can help you get higher accuracy, lower latency, and save time and money to spend on your company’s core competencies. <h3>Why Scale?</h3> Scale Document AI supports customers with quickly deploying a complete document processing solution with guaranteed quality SLAs for any of their document types. We use our own in-house OCR engine for identifying the text on a page, but don’t stop there. What’s unique about our approach is that we leverage base models built on state of the art Computer Vision and Natural Language Processing research. These models are trained on millions of data points and we further refine them for each customer use case. This fine-tuning uses Scale’s proprietary data labeling and ML training infrastructure that we’ve developed over the past few years to deliver a solution that consistently performs at high accuracy – even across unique edge cases, customer requirements, or variable document types. Scale provides Adaptive ML document processing for many enterprises and organizations across financial services, logistics, manufacturing, healthcare, real estate and government. For example, Scale delivers fast extraction from messy invoices <a href="http://scale.com/customers/brex" rel="noopener noreferrer" target="_blank">for Brex</a>, accurate and timely data from complicated shipping paperwork <a href="http://scale.com/customers/flexport" rel="noopener noreferrer" target="_blank">for Flexport</a>, and essential data out of titles, mortgage applications, and deeds <a href="https://scale.com/customers/states-title" rel="noopener noreferrer" target="_blank">for Doma</a> in real estate transaction processing. If you’re looking for a solution that’s higher quality, lower latency, and free of the hassle of creating and maintaining templates, <a href="https://scale.com/document-ai" rel="noopener noreferrer" target="_blank">learn more and contact us</a>.

OCR and even Intelligent Document Processing fall short, while AI-based extraction delivers.

With adaptive ML models performing extraction, Scale Document AI goes far beyond OCR.

Comparing Scale Document AI to Traditional OCR

Many industries that impact our day-to-day lives, including logistics,
manufacturing, financial services, insurance, and government, still run on
paper. These documents contain critical information that powers important
workflows like clearing shipments past customs, processing insurance claims,
underwriting loans, tracking machinery parts, issuing tax refunds, and parsing
clinical lab reports. Unfortunately, document processes still involve people
manually keying in information into digital systems. These labor-intensive
workflows significantly increase the time it takes to extract information,
leading to added costs, poor customer experience, and lack of scalability for
high volumes of documents.
&nbsp;
To tackle this challenge, companies previously used technologies like Optical
Character Recognition (OCR). However, OCR solutions only transcribe text
without extracting the most relevant fields. Most &ldquo;intelligent&rdquo; extraction
solutions rely on customers to manually define templates or hard-coded sets of
rules to handle data extraction. The rules dictate which part of the document
corresponds to a certain field, which means the system can only process the
document layouts that fit the original template. But, if you change vendors,
document types, or update the document in any way, then you need to manually
define another set of rules. Solutions like these take a lot of upkeep and
result in poor quality when dealing with complicated, variable document types.
&nbsp;
Imagine being able to feed any type of document into a system that can quickly
and accurately extract fields and link entities, even from new layouts or
formats &mdash; all without any setup or maintenance effort on your end.
&nbsp;
Tech-forward companies are recognizing the importance of deploying a robust
solution for automating entity extraction and linking because high quality,
structured data can:
<ul>
<li>Improve existing services by making them faster, cheaper, and more accurate</li>
<li>Enable new products based on the data unlocked.</li>
</ul>
&nbsp;
The challenge is building a system that can adapt to real-world variations,
especially across semi-structured and unstructured documents without
sacrificing accuracy. Here is how we do it.
<h2>Scale Document AI</h2>
&nbsp;
Scale Document AI relies on our
latest technology, Adaptive AI, to deploy refined machine learning models for
customers who demand high quality and low latency when it comes to document
processing. We leverage base models trained on millions of data points, and
further refine those models for each customer use case. This enables us to
extract and link entities from highly variable documents in seconds without
putting the burden of setup on the customer.
&nbsp;
What&rsquo;s unique about Adaptive AI is that we deliver a solution tailored to each
use case to extract the data our customers need at high quality &mdash; regardless
of any changes in document layouts. Unlike existing solutions, our machine
learning models thrive on challenging and varied documents by parsing the
structural layout of pages, contextualizing the meaning of words, and
understanding the relationships between different fields. We developed
Adaptive AI to actually understand the structure and the form fields&rsquo; meaning,
rather than simply learning where on a document to find a field (e.g.
understanding the vendor name instead of instructing that it is usually on top
left of document).
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/7356c354-9a87-455d-d0e7-6618e6d73100/public" alt="Scale Document Workflow">
&nbsp;
With Scale Document AI, you get:
<ul>
<li>Fine-tuned document processing models trained to meet</li>
<li>your exact requirements. Achieve higher accuracy than off-the-shelf</li>
<li>solutions across a variety of document layouts.</li>
<li>&nbsp;</li>
<li>Guaranteed quality up to 99%+ and turnaround SLA times as fast as</li>
<li>seconds</li>
<li>with a fully automated or hybrid document processing workflow using our</li>
<li>API. Scale provides integrated models and optional human-in-the-loop</li>
<li>quality assurance workforce.</li>
<li>&nbsp;</li>
<li>Painless and quick setup so your teams can focus on what&rsquo;s</li>
<li>important.</li>
<li>No more configuring templates or fixing errors. Upload sample documents</li>
<li>and provide a list of fields that you need extracted. There is no further</li>
<li>setup or maintenance required to start processing documents.</li>
<li>&nbsp;</li>
<li>Dedicated machine learning and engineering support. Work</li>
<li>with a world class technical team of machine learning researchers and</li>
<li>engineers dedicated to your document processing needs. We can help you</li>
<li>adapt as you scale your business.</li>
<li>&nbsp;</li>
</ul>
&nbsp;
<h2>Real-World Impact</h2>
&nbsp;
Scale Document AI serves many industries across financial services,
insurance, real estate, logistics, manufacturing, energy, healthcare, and
government, across a variety of document types and use cases such as loan
origination, customer onboarding, invoice/receipt/bill processing, fraud
prevention, claims processing, title closing, shipment processing, and
clinical lab report extraction.
&nbsp;
&nbsp;
In logistics, we partnered with
<a href="https://www.flexport.com/" target="_blank" rel="noopener noreferrer">Flexport</a>, which is building the
platform for global trade, to significantly increase efficiency and reduce
costs in what used to be a manual effort for processing critical shipping
paperwork. We quickly deployed our Adaptive AI for important logistics
documents like Bill of Lading, Arrival Notices, and others to reduce data
extraction errors and improve compliance across Flexport&rsquo;s global trade
network. James Chen, Flexport CTO, explained the impact of our partnership
at
Scale Transform, our conference earlier this year:
&nbsp;
&nbsp;
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/a186b4ed-c72c-4575-e374-aeb715927c00/public" alt="James_Chen_Quote">
&nbsp;
&nbsp;
Another applicable area for Scale Document AI is financial services,
specifically bill pay services, loan origination, and claims processing.
<a href="https://www.brex.com/" target="_blank" rel="noopener noreferrer">Brex</a>, a $7.4B startup modernizing
banking for businesses, uses Scale Document to power its new premium bill
pay product. When customers submit invoices in the Brex app or forward an
email with an invoice to the designated Brex email, our Adaptive AI extracts
all relevant information from these invoices instantly and accurately to pay
the bill, at higher quality and faster than off the shelf solutions.
&nbsp;
&nbsp;
We are excited to reimagine document processing to enable faster and better
workflows for challenging document types where conventional approaches don&rsquo;t
hit the mark. If you want to achieve modern operational efficiencies through
AI and see tangible downstream improvements in business metrics, contact us
at documentai@scale.com or sign up on our
<a href="https://scale.com/sales" target="_blank" rel="noopener noreferrer">website</a>.
&nbsp;

Scale Document's Adaptive AI ensures fast, accurate document processing, removing costly errors through machine learning

Prathik Naidu

How ML Linting Meaningfully Improves Adaptive Document Processing at Scale

Comparing Scale Document AI to Traditional OCR

OCR Is Not Enough. Try Scale Document AI.