Conclusion
Seismic shift in the rate of artificial intelligence (AI) adoption
The Scale Zeitgeist AI Readiness Report 2022 tracks the latest trends in AI and ML readiness. Perhaps the biggest AI readiness trend has been the seismic shift in the rate of AI adoption. Moving beyond evaluating use cases, teams are now focused on retraining models and understanding where models fail. To realize the true value of ML, ML practitioners are digging deep to operationalize processes and connect model development to business outcomes.
This artificial intelligence report explored the top challenges ML practitioners face at every stage of the ML lifecycle as well as best practices to overcome some of these challenges to AI readiness. We found that most teams, regardless of industry or level of AI advancement, face similar challenges. To combat these challenges, teams are investing in data infrastructure and working closely with annotation partners. We also explored different tactics for model evaluation and deployment. Although deployment methods and tools vary widely across businesses, teams are evaluating business metrics, using A/B testing, and aggregating model metrics. Wherever ML teams are in their development, we can share knowledge and insights to move the entire industry forward.
Our goal at Scale AI is to accelerate the development of AI applications. To that end, this report has showcased the state of AI readiness, identifying the key AI trends in 2022 in terms of the challenges, methods, and successful patterns used by ML practitioners who actively contribute to ML projects.
“AI is a very powerful technology, and it can have all kinds of applications. It’s important to prioritize applications that are exciting, that are solving real problems, that are the kind of applications that improve people’s lives, and to work on those as much as possible.”
Ilya Sutskever — Co-founder and Chief Scientist, OpenAI
Methodology
This survey was conducted online within the United States by Scale AI from March 31, 2022, to April 12, 2022. We received 2,142 total responses from ML practitioners (e.g., ML engineers, data scientists, development operations, etc.). After data cleaning and filtering out those who indicated they are not involved with AI or ML projects and/or are not familiar with any steps of the ML development lifecycle, the dataset consisted of 1,374 respondents. We examined the data as follows:
The entire sample of 1,374 respondents consisted primarily of data scientists (24%), ML engineers (22%), ML researchers (16%), and software engineers (13%). When asked to describe their level of seniority in their organizations, nearly half of the respondents (48%) reported they are an individual contributor, nearly one-third (31%) said they function as a team lead, and 18% are a department head or executive. Most come from small companies with fewer than 500 employees (38%) or large companies with more than 25,000 employees (29%). Nearly one-third (32%) represent the software/Internet/telecommunications industry, followed by healthcare and life sciences (11%), the public sector (9%), business and customer services (9%), manufacturing and robotics (9%), financial services (9%), automotive (7%), retail and e-commerce (6%), media/entertainment/hospitality (4%), and other (5%).
When asked what types of ML systems they work on, nearly half of respondents selected computer vision (48%) and natural language processing (48%), followed by recommendation systems (37%), sentiment analysis (22%), speech recognition (10%), anomaly detection/classification/reinforcement learning/predictive analytics (5%), and other (17%).
Most respondents (40%) represent organizations that are advanced in terms of their AI/ML adoption—they have multiple models deployed to production and regularly retrained. Just over one-quarter (28%) are slightly less advanced—they have multiple models deployed to production—while 8% have only one model deployed to production, 14% are developing their first model, and 11% are only evaluating use cases.
Group differences were analyzed and considered to be significant if they had a p-value of less than or equal to 0.05 (i.e., 95% level of confidence).
About Scale
Scale AI builds infrastructure for the most ambitious artificial intelligence projects in the world. Scale addresses the challenges of developing AI systems by focusing on the data, the foundation of all AI applications. Scale provides a single platform to manage the entire ML lifecycle from dataset selection, data management, and data annotation, to model development.