experiment

Header

Footer

PreFooter

SectionTitle

PaginationButton

BlogPage

Trusted by world class companies, Scale delivers high quality training data for AI applications such as self-driving cars, mapping, AR/VR, robotics, and more.

AI & ML Blog | Scale AI

NextImage

AnnouncementContextProvider

Screen

plasmic-graphcms

scale.com (Production)

lottie-react

plasmic-nav

react-scroll-parallax-global

plasmic-embed-css

plasmic-query

plasmic-basic-components

plasmic-tabs

react-aria

plasmic-cms

react-chartjs-2

plasmic-strapi

react-slick

radix-ui

react-quill

Product

Company

General

People

Engineering

Every year, nearly 200,000 service members transition to civilian life. As a fighter pilot who served for over two decades in the U.S. Air Force, I know the challenges these men and women face. From adjusting to “normal” life, to mental health struggles, to determining a new career path, it can be overwhelming and difficult to navigate. One of my favorite things about Scale is our CEO’s hiring philosophy. Simply put, he says to hire people who give a shit. That philosophy has brought an amazing group of people to this company who not only care about technology, but also care about our country, our government, and our service members. As we prepare to celebrate Veterans Day, we wanted to share some of our Scale veterans’ stories to honor them and their commitment to our country, and to also offer advice for those navigating the transition from service to civilian life. <h1>What did you do in the military?</h1>Jason Schmid: I was a U.S. Army Military Intelligence Analyst, and deployed to Iraq twice. The first time was as a part of the Iraq Survey Group, determining the disposition of Iraq’s suspected weapons of mass destruction programs. On that mission, I was part of the team interrogating former high ranking Iraqi Regime officials. On my second deployment, I was embedded as a Tactical Intelligence Trainer in an Iraqi infantry battalion. Dan Tadross: I was an Air Traffic Control Officer responsible for coordinating aircraft requests and the deconfliction of airspace. In my last few years of Active Duty, I had the opportunity to serve as an Innovation Advisor to the Secretary of the Navy on how to integrate AI into Aviation Command and Control (specifically how to shorten planning cycles). I also had the privilege of being a founding member of the Department of Defense (DOD) Joint Artificial Intelligence Center (JAIC). During my two years at the JAIC, I ran a number of programs from supporting testing and evaluation to establishing Project Gargoyle for Force Protection. Brianna Hester: I was a Geospatial Analyst for 24 years. My most recent position was the Operations Superintendent of the European Partner Integration Enterprise (EPIE) where I managed a 40-person, international intelligence team operating a $5.7M Intelligence, Surveillance, Reconnaissance (ISR) coalition program for combat mission planning, execution, and overall intelligence gathering and research. I oversaw ISR mission operations and imagery exploitation of U.S. ISR platforms employed for collection of full motion video imagery in support of the DOD, NATO, and coalition forces. I deployed to Qatar and Afghanistan and also traveled to numerous other countries for my duties. Jacob Sheehan: I was a U.S. Army Special Forces Officer, also known as a Green Beret, where I deployed on combat missions to Iraq and Afghanistan and on training missions in India, Thailand, and Korea. In addition to leading operational teams, I also had an academic assignment at West Point where I taught economics courses, ran the graduate scholarship program (Rhodes, Marshall, Fulbright, etc.) and designed a two-semester critical thought curriculum. Matt Beck: I was commissioned as a Naval Officer in 2008. I spent my first two years as a Surface Warfare Officer on a destroyer. I was selected to attend SEAL training after that tour and completed training in 2012. I led teams of various sizes on deployments to Europe, Africa, South America, and the Middle East. Tyler Haley: I was a Special Operations Communicator in the U.S. Army for 10 years. I spent most of my career inside of Special Operations doing everything from tactical communications to designing and deploying data centers. I’ve spent a large portion of my career in support of overseas operations from Afghanistan, Iraq, Syria, to most of Central Africa. I was lucky enough to get to finish my career in doing what I love most, which is any and all things cloud! <h1>Why Scale?</h1>Jason Schmid: When I left government service, it was important to me to join a company that values its contribution to national security. All of the Scale senior leaders I spoke to in the interview process made it clear that Scale is dedicated to using its technology to serve the nation. I think it is hard to overstate how important the work Scale is doing for the nation, both on the commercial and federal sides of the business. Dan Tadross: One of the most challenging aspects for the U.S. government to adopt and implement AI responsibly is the lack of structured data. Scale is at the forefront of developing structured data and can do so at a volume and quality that can help catapult government efforts and ensure the competitive edge for the U.S. Brianna Hester: I was intrigued by the job description. When I opened it and looked at the requirements, I had all the requested experience so I applied. After chatting with Austin Woo about some more specifics of the position, I knew I wanted to be a part of this program. Jacob Sheehan: I was primarily drawn to Scale through my hiring manager, David Kinsella. I was headed towards working at a financial technology company in my job search, but David introduced me to Scale’s compelling vision in the AI industry and the role it could play in making the U.S. government more nimble to respond to pressing challenges. He viewed my military experience as an asset and imagined my future impact at Scale rather than solely focusing on my non-traditional business operations background. Matt Beck: I was drawn to Scale’s mission of accelerating the adoption of AI. A lot of the work we are doing on the federal team has a direct application to our nation’s defense, and I wanted to be a part of that work. Tyler Haley: I love the team dynamics of start-ups. Everyone wears different hats depending on the meeting or the time of day. This is super relatable to being on small unit teams inside of the Special Operations community. Scale, in particular, pushes the boundaries inside of technology and solves hard problems, which is exactly what I did in the U.S. military. <h1>What do you do at Scale?</h1>Jason Schmid: I am Scale’s Head of Government Relations. I work to help Congress understand AI and ML capabilities so that they can build programs and policies that will help the U.S. government accelerate the adoption of AI. Dan Tadross: I am an Engagement Manager helping to bridge the gap between U.S. government requirements and experimentation and Scale’s expertise. Brianna Hester: I am the Synthetic Aperture Radar (SAR) Imagery Intelligence Subject Matter Expert (SME). Jacob Sheehan: I’m a Business Operations and Strategy Manager that supports the federal team. My main projects include growing a compounding network of labeling experts and labeler performance management. Matt Beck: I am a Federal Deployment Strategist focused on developing solutions to support the modernization of government systems. Tyler Haley: I am the Infrastructure and Platform Lead for our federal team. I partner closely with our software engineering and security teams to help design infrastructure that works on various AWS classification regions. <h1>How was your transition from service to civilian life?</h1>Jason Schmid: I transitioned from the Army 15 years ago, and the circumstances of my transition were pretty unique. I went from being in literal combat to being a civilian in about three weeks. I was very fortunate to have a job lined up already, but it would have been much better if I had time to take advantage of all of the resources available to transitioning service members. Such a jarring transition was not easy. We now know so much more about successfully transitioning veterans back into civilian life that would have made my transition so much smoother - not just for me, but for my family as well. Dan Tadross: I transitioned off Active Duty service after 12 years during the height of the pandemic in 2020. So the transition process was stressful and challenging, but when I eventually landed at Scale my level of apprehension subsided. Brianna Hester: My transition was pretty easy, as I was ready to begin my post-military life. The hardest thing to get used to was not having to tell someone my every move. Taking PTO without telling someone the address of where I would be for the duration of the PTO was a little unnerving the first time. Jacob Sheehan: I gave myself sufficient time, but there seems to be no such thing as too much time when it comes to major life transitions, especially if you have a family to take care of! I’m glad I had the space to think about what I wanted to do next instead of having to manufacture it quickly. Transitioning during COVID was a challenge from the standpoint of young kids being around all the time, so I would recommend finding a quiet space to do your transition thinking. Matt Beck: I had a wonderful transition departing the military. I met Shands Pickett early on in the process and kept in touch as positions at Scale became available. My preparation and exploration of many different post-military careers went well and resulted in me working at Scale. Tyler Haley: Transitioning was a bit tough for me. I got out of the service in May 2020 right as the pandemic was really setting in. Transitioning from being in a room with people that you are close with to only seeing people via Zoom and Slack was difficult at first. I was accustomed to seeing my daily contributions make a real impact, but when I got out, that faded away rather quickly. The good part about transitioning was being able to be at home and start a family. My wife and I welcomed our first kiddo into the world shortly after I got out. It gave me a new purpose and perspective in life. It helped me fill the void that the military left. <h1>Can you speak to your experience as a veteran at Scale?</h1>Jason Schmid: I think one of the best qualities of Scale’s culture is a genuine curiosity and passion to solve hard problems as a team. I think anyone who has served can connect with that. Dan Tadross: One of the aspects I miss most about serving in the U.S. Marine Corps is the comradery and sense of purpose. While nothing can compare with serving in a Marine unit, Scale has a culture that shares some similarities and the team I am a part of is motivated to succeed and support the U.S. government toward the responsible adoption of AI. Brianna Hester: Scale has become my new family. I still have my military family and always will but now I have added another branch to my family tree with everyone from Scale. They may be very different families but I wouldn’t trade either of them for anything and I can’t wait to see how much we accomplish together! Jacob Sheehan: Spending time with the many veterans on the federal team has maintained some of the camaraderie I enjoyed in the U.S. Army. I’m also pleasantly surprised how genuinely curious Scaliens across all of Scale are in veteran experiences! Matt Beck: Scale is extremely supportive of its veterans. My work keeps me engaged on a mission I believe in. Furthermore, Scale is extremely flexible in letting me maintain my connection to the U.S. Navy by supporting my drill requirements. Tyler Haley: Although I am still new to the team at Scale, I felt value-added from day one. As I mentioned earlier, feeling like what you do matters is a huge point of pride for most veterans. I instantly felt as if I was a part of the team and also was surrounded by other team members that have served. At Scale, you really get the experience of being back on those small teams that have a mission and purpose. <h1>What advice would you share with veterans making the transition?</h1>Jason Schmid: Take advantage of every resource available to support your transition, whether you think it applies to you or not, because you’ve earned it. It’s normal for recently separated veterans to feel some loss of purpose. It’s important to find ways to continue serving something larger than yourself, even if it's not part of a career path. Dan Tadross: There are a lot of opportunities for those who want to find alternative ways to continue to support the U.S. government. Many of the skills I learned in the U.S. Marines continue to help me to succeed in the private sector. Those experiences and capabilities that made you successful as a service member are in demand in the private sector, so when making the transition, don’t forget the value you bring to any organization. Brianna Hester: My recommendation would be to take a break and relax for a minute before jumping right into another job, if you can. It’s a huge change going from complete structure in your everyday life to no structure, and it can be hard on some people. Enjoy a small (or even large) break and use that time as a way to prepare yourself for the great things to come. Jacob Sheehan: Spend a lot of time exploring the industry you want to work in, understanding the companies uniquely positioned in that industry, and thinking about the strengths you can bring to roles in those companies. I’ve had many veteran friends get it backwards by over-indexing on the role (title) and barely giving any thought to the company or larger industry. Matt Beck: Always seek to improve your position as you look for opportunities. Forecast your worst case scenario out as far as possible. Once you are comfortable with your worst case scenario, seek out the roles and companies that you are most excited about and get connected early. Tyler Haley: Figure out what fills your voids. Find a routine and stick to it. A lot of the service members that I speak with feel the same way. Whether that’s getting up every morning and hitting the Peloton or cooking dinner for the family every night, find something that you can do for YOU. Find a new skill or hobby and try to master it. Get out of the house and away from the keyboard. Find time in your days to step away and get lunch with another veteran. Transitioning is different for everyone. Make time to see those who understand what you are going through. If you are interested in joining our team, check out our <a href="https://scale.com/careers#jobs" rel="noopener noreferrer" target="_blank">open positions</a>.

As we prepare to celebrate Veterans Day, we wanted to share some of our Scale veterans’ stories.

Scale Team Spotlight: Veterans at Scale

<h1> </h1><h1> Why build an ML-based workflow for data labeling?</h1> Delivering high-quality data labels quickly is a challenging task. Even for humans, image and video data commonly found in self-driving or robotic applications can be compositionally challenging to parse and correctly annotate. At Scale, we use machine learning to augment human workflows to both improve the quality and speed of labeling. Since deep learning models can struggle to perform consistently across diverse data domains like self-driving car scenes, finding the optimal balance between ML automation and human supervision is necessary to ensure we maintain high-quality.Autosegment, an ML-powered tool that can accurately segment instances within a human-defined bounding box, allows us to delegate high-level reasoning about semantics and taxonomy to Taskers (human labelers) and basic pixel-level processing to an ML model. This division of labor enables us to build a general purpose and accurate ML labeling tool. Despite using a simple neural network, Autosegment outperforms prior tools.<h1> </h1><h1> The Challenge of Semantic Segmentation Labeling</h1> Semantic segmentation involves human Taskers identifying and labeling every pixel in a visual scene — a notoriously time-consuming and tedious process. Most Tasker time in this manual approach — about 95% — is consumed by time-intensive work, such as carefully tracing the outlines of objects in a scene. For example: it takes roughly one minute to trace the outline of a car and only three seconds to click on that outline and select the label “Car.” What if we could instead use machine learning to automate the most laborious parts of semantic segmentation? What if an ML model did the semantic segmentation, and the Tasker only had to select the right label?<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/72c723b3-46ae-4384-b75b-4a87b5e40b00/public" alt="Semantic Segmentation Task"> A visual scene which has been semantically segmented. Each pixel is color coded to denote the class of object it represents, i.e. Pedestrian, Road, Bike, etc. As ML engineers, we rely heavily on quantitative validation metrics, such as label accuracy and intersection-over-union (IoU), to determine the quality of our models. However, when building models that optimize for a particular business outcome — for instance, reducing the time it takes Taskers to semantically segment images — we need to think critically about which model to use and how we measure its performance.. In this example, our objective is to produce predictions that streamline the Taskers’ job. This is not necessarily the same as “best model performance.” For example, an instance segmentation model might be consistently inaccurate by just a few pixels around an object. This model might have a high IoU score but very little practical usefulness because Taskers would need to re-do every trace of the outline. Conversely, a model that can perfectly segment an object, even if it only identifies half of it, would still be extremely useful. Despite having lower IoU validation scores, it’s still an improvement to the overall workflow. Because it takes so long to trace outlines manually, having a model produce pixel-accurate partial outlines is still a significant time saver. <h1> </h1><h1> The Active Tooling Paradigm and the Autosegment Tool</h1><h1> </h1>Autosegment is a machine learning tool that segments the most salient object within a human-provided box annotation. It incorporates Tasker feedback at annotation time in a way that is flexible and iterative. When building Autosegment, we realized early on that the only way to provide flexibility and high quality was to give Taskers access to Active Tools. Active Tools let Taskers make changes to the model inputs and parameters in real time to generate the best output for their use case. This is similar to “prompt engineering,” the process by which we can coerce a neural language model like GPT-3 to generate desired text when given precise prompts. For more information, OpenAI hosts <a href="https://beta.openai.com/docs/guides/completion/prompt-design" rel="noopener noreferrer" target="_blank">excellent documentation</a> on how to design prompts for their GPT-3 model. <img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/07588dc3-6a06-45e7-c61f-a51562470900/public" alt="Open-AI GPT-3 Model"> Humans working alongside models can produce better results faster than models alone Using Active Tools, we give Taskers as many options as possible to make their tasks easier. For example, they can adjust the annotation box shape and the prediction confidence threshold to configure the model to speed up their particular labeling task. <img src="https://cdn.builder.io/o/assets%2Fe0438815ba51486bbb6a202747122d4b%2Ffef154571d5846b6b87674a46a123ac5%2Fcompressed?apiKey=e0438815ba51486bbb6a202747122d4b&amp;token=fef154571d5846b6b87674a46a123ac5&amp;alt=media&amp;optimized=true" alt="Autosegment Tool Demo">A demo of the Autosegment tool. Enabling humans to freely explore the output space of the model is a key advantage of active tooling. We've already seen how this is useful. Overall, we have observed the median annotation time per scene is about 30% faster with Autosegment. We also interviewed Taskers to see how they are using the tool. While our model is trained to segment cars and motorcycles, one Tasker used the tool to segment everything. While this wasn’t a workflow we had envisioned, it halved their time per scene segmentation, from 40 minutes to 20 minutes. In a discussion we had with five Taskers, they explained that they consider Autosegment to be more accurate and efficient than previous ML solutions. Now we can simply supply robust and general online models and let Taskers optimize the rest. <h1>The Autosegment Model</h1>Autosegment is a convolutional neural network trained to segment the most salient instance within a Tasker-defined bounding box. This simple design allows us to significantly reduce our customers’ operational overhead to deploy ML, while providing pixel-perfect instance segmentations, even on previously unseen instance types. The Autosegment model does not have to be particularly complex to produce compelling results. Autosegment has no notion of different semantic types; instead, it abstractly identifies and segments based on scene composition. The model is trained to always generate a segmentation. The training data includes a few key classes like pedestrians and vehicles. The task is binary; it simply determines whether each pixel belongs to the salient object or not. We don’t train the model with negative samples. Instead, the model makes its best guess on each pixel to generate a segmentation. We define the salient instance as the object that owns the central pixel of the crop, so even if the scene is cluttered, the model can identify the target instance to segment. This binary segmentation task across instance classes and types allows the model to generalize well on new classes and taxonomies. Indeed, we can transfer not only between objects of the same class, such as cars in different countries, but between different classes altogether — for instance, moving from cars to motorcycles to street signs. This transfer can be achieved with limited retraining or, in some cases, no retraining at all, achieving zero-shot transfer. <iframe class="ql-video" frameborder="0" allowfullscreen="true" src="https://scaleai.wistia.com/embed/medias/1a4x5jij9v"></iframe> Using salient object detection on specific crops of the scene can generate even more accurate segmentation. We were able to double the edge precision score from 0.26 to 0.52 when compared to off-the-shelf full-scene semantic segmentation models. Instead of supplying a downsampled version of the whole image to a neural network, the selected region is fed to the model at full resolution. Because the input is smaller, fewer pooling operations are required to achieve a globally receptive field. As a result, there is less downsampling and blurring. Our model achieves better fidelity around the edges of instances, a crucial part of providing high-quality data annotations to our customers. Improving full-scene labeling to the same degree would require exponentially more effort and complexity. In summary, we developed Active Tooling to create an ML-based workflow capable of solving basic computer vision problems. This workflow helps our Taskers reach higher levels of quality faster than in previous workflows. Innovations like Autosegment are critical to enable the “Moore's Law of data labeling.” At Scale AI, we’re excited to push the boundaries of human-AI interaction for our customers.

Scale AI accelerates image annotation with semantic segmentation active tooling. 

Reducing image labeling time by up to 30% and doubling segmentation quality.

Faster Semantic Segmentation with Machine Learning-Assisted Workflows

AI is no longer the exclusive domain of large enterprises. It&rsquo;s entering the business mainstream&mdash;but it&rsquo;s successful only sometimes, says Salesforce President and Chief Operating Officer Bret Taylor. A thoughtful AI strategy should be customer-focused, with clearly defined business outcomes, and the key to all of this is data.
In his talk at the <a href="https://scale.com/events/transformx" target="_blank" rel="noopener noreferrer">Scale TransformX 2021 conference</a>, Taylor discussed the key considerations that businesses of all sizes should keep in mind to ensure that they delight their customers and achieve sustainable business outcomes.
<iframe class="ql-video" src="https://fast.wistia.com/embed/medias/42dyrzqs4c" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
Here&rsquo;s a summary of the seven key takeaways from Taylor&rsquo;s presentation. For more details, watch the session video above.
<h3>Business Today Is All About the Data</h3>
&ldquo;If you don't have a digital business, you don't have a business,&rdquo; says Taylor.
Enterprises need to maintain constant engagement with customers, supply chains, partners, and employees, in granular detail. Each of these interactions produces data that businesses can use to create more intelligent and personalized relationships. This <a href="https://insidebigdata.com/2017/02/16/the-exponential-growth-of-data/" target="_blank" rel="noopener noreferrer">growing tsunami of data</a> requires intelligent algorithms and automation to make sense of it all. As Marc Andreessen, co-founder of venture capital firm Andreessen Horowitz, famously said, <a href="https://a16z.com/2011/08/20/why-software-is-eating-the-world/" target="_blank" rel="noopener noreferrer">&ldquo;Software is eating the world.&rdquo;</a>
Today, however, it&rsquo;s AI that has become central to everything that a company such as Salesforce does.
<h3>Build a Data Culture</h3>
AI and deep learning can&rsquo;t be the exclusive domain of technology companies. To get there, every company must strive to <a href="https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/McKinsey%20Analytics/Our%20Insights/Why%20data%20culture%20matters/Why-data-culture-matters.ashx" target="_blank" rel="noopener noreferrer">develop a data culture</a> that allows employees to see and understand the flow of information moving through the business. A <a href="https://www.gartner.com/smarterwithgartner/do-you-need-a-chief-data-officer" target="_blank" rel="noopener noreferrer">chief data officer (CDO)</a> should act as a steward for all that information, without trapping data in internal silos. Instead, everyone needs to be empowered to make data-informed decisions based on objective views of customers, supply chains, and the flow of resources.
Ultimately, companies that create open data cultures will easily outcompete those that don&rsquo;t.
<h1>Ethical AI Builds Trust</h1>
Shifting toward a data culture requires building in <a href="https://www.salesforce.com/blog/what-is-ethics-by-design/" target="_blank" rel="noopener noreferrer">ethics by design</a>, in much the same way that enterprises have incorporated <a href="https://patchstack.com/security-design-principles-owasp/" target="_blank" rel="noopener noreferrer">security by design</a>.
To deliver an ethical deployment of AI, the number one value must be trust. With that in mind, Salesforce created the position of <a href="https://www.cnbc.com/2018/12/10/salesforce-hires-its-first-chief-ethical-and-humane-use-officer.html" target="_blank" rel="noopener noreferrer">chief ethical and humane use officer</a> to guide the company&rsquo;s adoption of AI.
Salesforce wanted to ensure that all of its AI is responsible, accountable, transparent, empowering, and inclusive. Taylor contends that failing to take this into account from the start inevitably leads to unintended negative consequences and general degradation of trust in technology.
Society is now holding software and AI to a higher standard. Companies that develop AI today must embrace conversations about ethics and trust, and they must embed those conversations into their development processes. Building trust also requires bringing in stakeholders who aren&rsquo;t data scientists to help shape the responsible use of these technologies.
<h3>Focus on Strategy, Not AI for AI&rsquo;s Sake</h3>
While every company needs to become an AI company to remain competitive, too many organizations are succumbing to the hype about digital transformation without having <a href="https://www.thetilt.com/content/stages-of-digital-transformation" target="_blank" rel="noopener noreferrer">clear goals</a> in mind. According to the <a href="https://www.gartner.com/en/information-technology/insights/cio-agenda" target="_blank" rel="noopener noreferrer">2021 Gartner CIO Agenda survey</a>, while 94% of CIOs are working to deploy AI, only a quarter have done so successfully.
&ldquo;Don&rsquo;t start with the technology; start with customers,&rdquo; Taylor says. Focus on their experience and what your customers need and want. Clarify the project&rsquo;s scope and look for the easy wins first. &ldquo;If AI becomes the purpose of the project, it becomes a bottomless pit of investment,&rdquo; he says.
<h3>One Shortcut to Good AI: Don&rsquo;t Build It; Buy It</h3>
Salesforce&rsquo;s <a href="https://www.salesforce.com/products/einstein/faq/" target="_blank" rel="noopener noreferrer">AI platform Einstein</a> is a prime example of Salesforce focusing on building AI technology that is customer-focused and that &ldquo;can be turned on with clicks, not code,&rdquo; says Taylor. Salesforce customer <a href="https://www.salesforce.com/blog/orvis-ai-customer-engagement/" target="_blank" rel="noopener noreferrer">Orvis uses Einstein</a> to do simple analyses of the potential engagement of its marketing messages prior to their publication.
That approach increased Orvis&rsquo; click-through rate by 22% and boosted traffic by 30%. This is one way an enterprise can deploy customer-focused experiences with the help of AI, without having to invest in the substantial skills and technology normally associated with developing it.
<h3>Align Your AI Investment with Business Priorities</h3>
A common key strategy, especially for B2C enterprises, starts with identifying use cases that increase personalization and automation, thus streamlining the customer experience. Then, identify the data and AI tools that could help create those use cases.
For its part, Orvis&rsquo; goal is to keep its support calls short, and <a href="https://www.salesforce.com/news/press-releases/2019/11/19/hey-einstein-salesforce-brings-voice-to-every-customer-experience/" target="_blank" rel="noopener noreferrer">Einstein can help by transcribing and analyzing customer questions</a> and suggesting answers to call center personnel in real time.
The lesson here: Bring in just a few machine learning experts and combine them with off-the-shelf SaaS platforms to discover unique value in AI that can be customized around your business.
<h3>Don&rsquo;t Forget the Quick Wins</h3>
Getting value from AI isn&rsquo;t science fiction, Taylor says. Simply step back and focus on the low-hanging fruit to achieve tangible business outcomes.
<h3>Learn More</h3>
To learn more about how Salesforce built its AI strategy and lessons learned,&nbsp;<a href="https://exchange.scale.com/public/videos/enterprise-ai-strategy-bret-taylor-on-lessons-learned-at-salesforce" target="_blank" rel="noopener noreferrer">watch Taylor&rsquo;s talk and view the full transcript here</a>.

Customer-first AI strategy starts with clear goals and reliable data — the key to driving real business outcomes.

AI Strategy: Data & Customer Focus

7 Elements of a Well-Thought-Out AI Strategy: Salesforce COO Bret Taylor

img_fallback.png

Dr. Li shares explores how recent AI research helps robots  perform both short-horizon and long-horizon tasks.

Dr. Fei-Fei Li On Teaching Robots Complex Tasks - TransformX 2021 Conference - Scale AI

On Teaching Robots To Perform Complex Tasks with Dr. Fei-Fei Li 

Today we are excited to welcome SiaSearch, a leading data management platform, to Scale. We acquired SiaSearch, a spin-off of the European AI venture studio Merantix, to enable us to expand Scale’s suite of tools for every AI team, broaden our footprint in European markets, and deepen the technical expertise of the company. We know that the autonomous vehicle (AV) industry is one of the best breeding grounds for AI technology. SiaSearch’s Berlin-based team brings experienced, high-caliber engineers from the European AV industry to Scale, which will help us deliver world-class AI solutions to support the complete machine learning lifecycle. The SiaSearch team will join Scale's Nucleus business unit as we grow our platform to support more data types and use cases across a wide array of industries and markets. SiaSearch’s unique interval querying engine, which is inspired by algorithms originally designed for large-scale analysis of DNA sequences, allows for lightning-fast search across massive video and LiDAR datasets and complements what Scale is building with <a href="https://studio.plasmic.app/scale.com/nucleus" rel="noopener noreferrer" target="_blank">Nucleus</a>. We are excited to weave the SiaSearch technology into the Nucleus product, accelerating development and improving the developer user experience. We see even more potential down the line to incorporate SiaSearch’s world-class team and technology throughout our entire suite of products. Merantix and the SiaSearch team have a strong network in Europe and have partnered with the world’s leading automakers, including Porsche and Volkswagen. We’re thrilled to expand Scale’s footprint to Germany and look forward to working with SiaSearch’s customers and networks to significantly increase our ability to serve European-based customers. We’re excited for what the future holds with this latest addition to our team, and we’re always looking for new opportunities. If you are interested in joining this incredible team, check out our <a href="https://scale.com/careers#jobs" rel="noopener noreferrer" target="_blank">open positions</a>. Or, if you’re interested in partnering with us, please <a href="https://scale.com/sales" rel="noopener noreferrer" target="_blank">reach out</a>.

Scale has acquired SiaSearch to further enhance Nucleus, its data management platform.

The SiaSearch team and Nucleus will collaborate on Dataset Managment for Nucleus.

SiaSearch Acquisition: Delivering the Full Data Engine to Our Customers

Waymo’s co-CEO Dmitri Dolgov shares Waymo’s approach to building powerful machine learning models for self-driving vehicles

Dmitri Dolgov of Waymo Discusses Data-centric AI - TransformX 2021 Conference - Scale AI

Dmitri Dolgov of Waymo Discusses Data-centric AI

The co-founder of Schmidt Futures and a former CEO of Google, discusses how AI will shape our global future. Eric was joined by Scale AI CEO Alexandr Wang

Eric Schmidt Discusses the Geopolitics of AI - TransformX 2021 Conference - Scale AI

Eric Schmidt Discusses the Geopolitics of AI

Justin Basilico, Director of Machine Learning and Recommender Systems at Netflix describes how ‘everything at Netflix is a recommendation’.

Netflix on Recommendations and Personalization - TransformX 2021 Conference - Scale AI

Netflix Explains Recommendations and Personalization

Clement Delangue HuggingFace CEO, shares what Transformers can do. He explores how we might ensure they are used ethically and transparently.

Clement Delangue HuggingFace CEO on Transformers - TransformX 2021 Conference - Scale AI

HuggingFace CEO, Clement Delangue: Are Transformers Becoming the Most Impactful Tech of the Decade?

Today we&rsquo;re thrilled to share that Nucleus now supports 3D Sensor Fusion data, including LiDAR point clouds.
<a href="https://scale.com/nucleus" target="_blank" rel="noopener noreferrer">Scale Nucleus</a> is the mission control for your data, providing advanced tooling for curating, visualizing and exploring machine learning datasets and debugging ML models. We recently launched an all-new Nucleus for dataset management, complete with privacy features and updated Autotag capabilities, all to help teams build better ML models by building better datasets. Now, Nucleus can help teams manage 3D datasets to develop highly accurate perception models for a range of use cases, including autonomous vehicles, robotics, and augmented/virtual reality (AR and VR).
Scale Nucleus helps you visualize and debug 3D Sensor Fusion data by making it easy to:
<ul>
<li>Construct Scenes that contain a sequence of LiDAR point clouds and their corresponding camera images over time</li>
<li>Visualize, query and share sensor fusion data with colleagues</li>
<li>Integrate Sensor Fusion data with your 3D model predictions to uncover and identify:</li>
<li>Discrepancies between 3D object detection model outputs and ground truth</li>
<li>False positive, common class confusions, and cuboid predictions that have low Intersection over Union (IoU) with ground truth</li>
<li class="ql-indent-1">Model performance and improvements over time</li>
</ul>
The Nucleus API makes it easy to import your data, construct scenes, and add model predictions. You can also import data from an existing Scale project directly into Nucleus.
<h2>A Primer: Nucleus Scenes</h2>
Scenes are the building blocks of 3D datasets in Nucleus and are designed with point clouds as the primary unit of data. More formally, a Scene is a sequence of Frames over time, where each Frame contains exactly one point cloud and an optional set of camera images. Here we can see a LiDAR point cloud above two paired images (the full dataset includes more than just front and back cameras, but a full suite of sensor data), which helps explain why we designed Nucleus Scenes to encapsulate data across both time and sensor modalities:
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/00157f16-6ed9-4ffc-45cc-60b676806e00/public" alt="Structure Diagram of Nucleus 3D Scenes">One useful way to interpret a LiDAR Scene is as a table of sensor readings. A row in the table corresponds to a sequence of readings from a particular sensor, while a column in the table represents a collection of sensor data from the same timestamp (we call this a &ldquo;frame&rdquo; of the scene).
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/6ea98ea3-b4d4-40eb-f36f-3562514cab00/public" alt="Complex Nucleus 3D Scene on SF's Embarcadero, Quarter Angle View">
On their own, point clouds can be difficult to interpret, due to lack of motion cues or associated camera data. Nucleus Scenes make sets of 3D and 2D data easier to interpret for the following reasons: first, any number of 2D camera frames can be attached to each 3D LiDAR frame; and second, Scenes link frames from the same timespan so that you can examine all frames in a linear fashion.
Let&rsquo;s now view the same Scene in Nucleus and see if we can identify the mystery object from above:
<iframe class="ql-video" src="https://scaleai.wistia.com/embed/medias/8ikm69knof" frameborder="0" allowfullscreen="allowfullscreen"></iframe>
After uploading predicted cuboids from an ML model for your Scene, Nucleus automatically computes performance metrics under the hood. You can sort predicted cuboids by IoU or filter by false positives to quickly narrow in on your model&rsquo;s failure modes. (These features have been available for 2D image analysis for some time, but today we are announcing their ability for 3D data, too.)
To summarize, by creating and uploading 3D Scenes to Nucleus, you can easily visualize LiDAR point clouds and images side-by-side, quality-check your ground truth annotations, and search your data based on metadata or even ML-produced attributes. Lastly, you can use Nucleus to measure key prediction performance metrics, such as IoU and false positives.
<img src="https://imagedelivery.net/wLbZE4_NzVVdgHc15St55g/6908e9df-249d-47df-4db5-a39827556c00/public" alt="Birds' Eye View of SF Embarcadero as Seen through LiDAR and Multiple Image Sensors in Nucleus Scene">
<h2>Getting Started with 3D Data in Nucleus</h2>
Visualizing, querying, and measuring the quality of sensor fusion datasets is a challenge facing even the most sophisticated ML teams. With the newly launched support of multimodal scenes, Nucleus makes it easy to navigate through massive datasets, and quickly narrow down on the most difficult examples. Unleash the full potential of your datasets by leveraging Nucleus for debugging, search, and quality related workflows.
The complete code for this tutorial is available in <a href="https://colab.research.google.com/github/brwillia/nuc-3d/blob/main/nucleus_3d_demo.ipynb#scrollTo=rr88VqEHaL2_" target="_blank" rel="noopener noreferrer">this Colab notebook</a>. If you&rsquo;d like to follow along step by step and create a personal dataset on Nucleus, you will first need to make a <a href="http://scale.com/signup" target="_blank" rel="noopener noreferrer">free Scale account</a> and obtain a test API key (which you can <a href="https://dashboard.scale.com/settings/profile" target="_blank" rel="noopener noreferrer">find here</a> if you navigate to your user icon, and then click on API keys). You can then copy the notebook and add your personal key to upload your data and try out Nucleus for yourself. At the end, you&rsquo;ll also find a section that shows you how to format and upload your own LiDAR data for curation and management in Nucleus if you&rsquo;re ready to manage your own 3D data in Nucleus.

Learn how to ingest and curate 3D LiDAR point cloud data in Nucleus.

Nucleus now supports 3D Scenes.

Managing 3D Sensor Fusion Data with Scenes in Scale Nucleus

URL redirections for Website. Disclaimer: takes 5~ min for each added or edited.

url-redirects

Use relative path. use `/blog/foo` not `https://scale.com/blog/foo`

blog-filter-type

homepage-quotes

logos

homepage-content

quotes

blog-category-model

announcement-banner

homepage-resources

custom-fonts

homepage-event

author

Event model for new marketing events page

event

Optional second image intended to show on the grid view, should be a more squared image closer to 1:1 aspect ratio than the one used in the Cover field.

guides

announcement-news

homepage-research

legal

slugs must be all lowercase. ie: ‘my-slug’

eval

datastream

summit-2023-speakers

customers

Display order on the page /customers. Higher values show first

summit-2023-agenda

blog-model

Option to list or not list this post in the Blog index. Enabled = not list in the index, Disabled = list in the index

summit-2023-partners

enterprise-use-cases-categories

enterprise-use-cases-categories-items

partners

data-samples

enterprise-use-cases

Use only one image or video option. Add another item to add another source.

collapsible-sections

research

basecamp-speakers

Example: https://www.twitter.com/username

Example: https://www.linekin.com/in/username

leaderboards

Text for a short description like: Deprecated (as of January 2025)

Keyword for search. Add multiple keyword with comma. Example: hle, hleto, vista, enigma

Blog