Scale collaborates with NVIDIA to power the next generation of LLMs with NeMo SteerLM
Scale AI is proud to collaborate with NVIDIA to power the next generation of LLMs for generative AI. By leveraging Scale’s high-quality training datasets and NVIDIA NeMo SteerLM - a simple LLM alignment technique that allows dynamic steering of models during inference - developers can create applications for a variety of enterprise use cases including education, chat bots, and gaming.
High-quality data is a critical resource in training the world’s leading aligned models, including for techniques like SteerLM. Scale’s Data Engine powers the most advanced LLMs with world-class RLHF, expert-generated data and model test & evaluation. Scale produces more RLHF data than anyone else: on track for two million hours (228 years) of data in 2023.
By combining NVIDIA’s latest research in generative AI with Scale’s datasets, this collaboration provides enterprises with high-quality data that can be used to align LLMs to follow instructions. SteerLM demonstrates new capabilities in education, enterprise, gaming, and more. For example, SteerLM enhances education by tailoring language model responses to individual learning preferences such as verbosity and complexity. For gaming, SteerLM strengthens non-player characters (NPCs) by enabling customizable personalities for immersive, human-like interactions during gameplay.
To support generative AI advancements, NVIDIA and Scale have open sourced the dataset. This dataset contains 37k samples with various dimensions including helpfulness, correctness, complexity and verbosity. Releasing this evaluation data enables the broader research community to develop language models and continue innovating research. For example, enterprise software developers can leverage NVIDIA’s open source toolkits and advanced models across a variety of industry applications, without having to build LLMs from scratch.
Use Cases for NVIDIA SteerLM
Powered by Scale's training data, NVIDIA SteerLM technique can unlock new capabilities in education, gaming, enterprise chat bots, and more. Below, we’ve shared a few early use cases for SteerLM. We look forward to seeing developers customize models to fit their own needs across many more applications.
Education: SteerLM can be used to customize LLM responses based on the personalized needs and preferences of teachers and students. Every student learns at their own pace and is critical to achieving learning outcomes both inside and outside of the classroom. SteerLM has knobs for Complexity and Verbosity of responses, which can be used to adjust how teachers and students would like the LLM to answer their queries. In addition, SteerLM can be trained to be more factually correct and coherent, making it suitable for education settings.
Gaming: NVIDIA integrated SteerLM into an AI system for more interactive and immersive non-playable character (NPC) experiences. SteerLM strengthens AI NPCs by enabling developers to customize their personality for more emotive, realistic dialogues and emotions. Specifically, SteerLM trains models that align generated text with adjustable attributes, allowing developers to easily modify model behavior through attribute sliders rather than full model retraining.
Retail: SteerLM can be used to power chatbots that generate responses tailored to different groups of retail customers. For example, some customers may prefer more detailed explanations, while other shoppers may want shorter answers. SteerLM enables the chatbot to dynamically tune its responses for each specific user.
Looking Ahead
Our SteerLM collaboration demonstrates the immense potential of combining the latest advancements in generative AI with expert-curated datasets. As NVIDIA and Scale continue to partner on additional data initiatives, including multi-turn retrieval augmented generation (RAG), we hope to unlock even more powerful enterprise applications across industries.
Generative AI is evolving rapidly. As techniques like SteerLM become more capable, the need for high-quality training datasets will continue to grow to build better models. Through our world-class Generative AI Data Engine, Scale is committed to delivering the best datasets tailored to your specific needs.
Together, NVIDIA and Scale are excited to open source high-quality datasets that can be used to train LLMs. We look forward to empowering more organizations to leverage NVIDIA’s managed services and models to transform their industries and shape the future of AI.
Learn more about NVIDIA’s SteerLM dataset by checking out the research paper and dataset. Get started with SteerLM today using NVIDIA NeMo, an end-to-end framework for building, customizing and deploying generative AI models anywhere.
Learn more about Scale’s Data Engine and how it can power your team’s model development.