Spencer Papay

Strategic Product Manager

On November 28th, OpenAI released a new addition to the GPT-3 model family: <code>davinci-003</code>. This latest model builds on <a href="https://openai.com/blog/instruction-following/" target="_blank" rel="noopener noreferrer">InstructGPT</a>, using reinforcement learning with human feedback to better align language models with human instructions. Unlike <code>davinci-002</code>, which uses supervised fine-tuning on human-written demonstrations and highly scored model samples to improve generation quality, <code>davinci-003</code> is a true reinforcement learning with human feedback (RLHF) model. It is trained with <a href="https://openai.com/blog/openai-baselines-ppo/" target="_blank" rel="noopener noreferrer">PPO</a> to optimize the generated text&rsquo;s score against a separate &ldquo;reward model&rdquo;, which is trained on rating comparisons by human graders of different model outputs. More details can be found in OpenAI&rsquo;s <a href="https://beta.openai.com/docs/model-index-for-researchers" target="_blank" rel="noopener noreferrer">model index for researchers</a>. The net result is that <code>davinci-003</code> is tuned to produce outputs that it thinks would be scored highly by humans. Scale is proud to partner with OpenAI to provide this human feedback.
OpenAI&rsquo;s announcement email mentions the following improvements for <code>davinci-003</code>:
<ul>
<li>&ldquo;It produces higher quality writing. This will help your applications deliver clearer, more engaging, and more compelling content.</li>
<li>It can handle more complex instructions, meaning you can get even more creative with how you make use of its capabilities now.</li>
<li>It&rsquo;s better at longer form content generation, allowing you to take on tasks that would have previously been too difficult to achieve.&rdquo;</li>
</ul>
But how much better is <code>davinci-003</code>, really? We decided to put it to the test quantitatively. Using <a href="https://scale.com/spellbook" target="_blank" rel="noopener noreferrer">Scale Spellbook</a>, the platform to build, compare and deploy large language model apps, we evaluated <code>davinci-003</code> versus <code>davinci-002</code> across tasks ranging from few- and zero-shot classification, summarization, and poetry writing. Here&rsquo;s what we found.
<h3>Classification</h3>
It&rsquo;s well known that <code>davinci-002</code> performs very well on classification with a few-shot prompt, and it appears that <code>davinci-003</code> offers comparable (but slightly worse) performance few-shot. Using Spellbook&rsquo;s built-in evaluation feature on 250 Yelp reviews, we compare the two models&rsquo; classification of these reviews as Positive, Negative, or Neutral in a four-shot prompt&mdash;meaning four labeled examples are included in the prompt itself.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F65f8dfd9c9744c15967d03341c5cdc73">
The davinci-003 model has 87.2% accuracy on this 250 Yelp review dataset sample, slightly lower than davinci-002&rsquo;s 91.6%.
However, if we ask the two models to classify these same reviews in a zero-shot prompt, we find that <code>davinci-003</code> scores more than 30% higher in accuracy than <code>davinci-002</code>&mdash;and on par with a <code>davinci-002</code> few-shot prompt! This is a substantial improvement in zero-shot classification performance. In this classification setting, adding examples to the prompt actually lowers <code>davinci-003</code>&rsquo;s accuracy.
<img src="https://lh6.googleusercontent.com/Wr30hJNdDAekBQTxtBRBN40_GR1CIElZlygaJSrAbGd6OK20V-pGS_bI5BoCF2SS4az3z7EFPYde3k5TnoVKHZd-68dwlhtHctxB7XtW5zqOXOjQkY2P8SxCkAjzlmg1PpKQ1hzEgob2W18laWD8FfSjNYShqjurevSldtMENyh8DUALgcMeYi4HQKi-">
Beyond accuracy, we found that asking <code>davinci-003</code> to explain its rationale in a zero-shot prompt works reliably, and provides a better explanation than a one-shot <code>davinci-002</code> about 75% of the time, based on Scale&rsquo;s human evaluation workforce judging a sample of 100 outputs.
<h3>Text Generation</h3>
The most notable performance leap comes in how <code>davinci-003</code> understands word and sentence structure, especially rhyming and syllables. In prompting for a poem that rhymes, <code>davinci-003</code> constructs a desired output, whereas <code>davinci-002</code> largely fails to implement rhyming.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F31cb06da7e1a488a91f62ef14feee823">
<code>davinci-003</code> is also better equipped to handle requests for complex output structures, such as iambic pentameter (think Shakespeare). <code>davinci-002</code> certainly provides a poem, but ignores the request for lines of 5 pairs of stressed and unstressed syllables (buh-BUM). While not perfect, <code>davinci-003</code>&rsquo;s output is significantly more faithful to the requested prompt.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fc3fc8edf1aea432bb7bb5118a6dc8c82">
<h3>Summarization</h3>
One practical difference between the two variants is verbosity. In our experimentation, <code>davinci-003</code> consistently generates longer outputs than <code>davinci-002</code> on the same prompts. This is most visible in summarization tasks, where the value of the result may depend on its brevity.
In a sample of 60 articles, we tested both variants&rsquo; summarization capabilities by asking them to summarize the introductory paragraphs of lengthy Wikipedia pages. The additional text generated by <code>davinci-003</code> is typically beneficial, including information central to an article&rsquo;s importance. However, the resulting summaries are also wordy. In our analysis, <code>davinci-003</code> generates outputs about 65% longer than <code>davinci-002</code> under identical prompts.
<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fbbd0ed222dff4216ab9173d07d176890">
It will be important for GPT-3 users to bear in mind <code>davinci-003</code>&rsquo;s talkativeness when choosing between the two variants. In our testing, given the task of generating a 1-sentence summary, <code>davinci-003</code> produced 42 words on average, nearly doubling <code>davinci-002</code>&lsquo;s average of 23. The shortest summary <code>davinci-003</code> gave back was a beefy 32 words, compared to only 12 words for <code>davinci-002</code>. This tendency extends to multi-sentence summaries as well.
For use cases that value brevity, <code>davinci-002</code> is the safer choice. Nevertheless, for many use cases, the longer and more detailed completions provided by <code>davinci-003</code> can be beneficial.&nbsp;
<h3>Conclusion</h3>
Overall, <code>davinci-003</code> is a meaningful improvement from <code>davinci-002</code>, especially for zero-shot classification and complex text generation.
While many of the benefits of the latest iteration of GPT-3 are apparent at the surface, concrete evaluation metrics ensure models and prompt strategies are performing as well as possible.
Luckily, Spellbook makes that easy, allowing users to run instant programmatic evaluation in classification use cases, or sending model outputs to Scale&rsquo;s high-quality workforce to get quantitative evaluation metrics for generative use cases.
<a href="https://scale.com/spellbook" target="_blank" rel="noopener noreferrer">Sign up for Spellbook today!</a>
P.S. If you&rsquo;re looking for a little dramatic flair in your emails to coworkers, <code>davinci-003</code> has you covered. It performs significantly better at creatively translating text or even explaining code as different celebrities in a zero-shot setting.
&nbsp;
BranchCommitmaintestmaintest <img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fa4f4e03e1cef4d28a44fcf465150aeca">
<pre class="ql-syntax" spellcheck="false"> Write a short rhyming poem about winter in New York davinci-002
davinci-003
</pre>
In New York City When winter descends
The air is so crisp and chilly
And the wind never ends
The snowflakes are so big and fluffy
You just want to grab a handful
And then make a snowball
People hurry to and fro
Bundled up in coats
Hats, and scarves, and gloves
No one wants to be caught
Lights are twinkling in the night, It's wintertime in New York City so bright.
The snow is glistening, the wind is cold,
The atmosphere is really bold.
The snowflakes fall with a gentle grace,
Adorning the city with a lovely lace.
The air is crisp, and the people merry,
It's a magical winter in New York City.
Text Generation
The most notable performance leap comes in how davinci-003 understands word and sentence structure, especially rhyming and syllables. In prompting for a poem that rhymes, davinci-003 constructs a desired output, whereas davinci-002 largely fails to implement rhyming.

We evaluate davinci-003 across a range of classification, summarization, and generation tasks.

Evaluating davinci-003: Performance, Improvements, and Gaps

How Much Better is OpenAI’s Newest GPT-3 Model?

Within the field of computer vision, semantic segmentation is a close approximation to describing the visual world the way humans perceive it: entities with precise boundaries. For use cases where nuanced scene understanding is mission-critical, like autonomous vehicles and intelligent robots, building models requires massive amounts of highly accurate segmentation masks with pixel-level precision.The exactness required near object edges that makes semantic segmentation valuable also makes it incredibly time-consuming for humans to annotate.&nbsp;Given taskers annotate thousands of images per day, we wanted to accelerate the <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">annotation</a> process without losing fine-grain precision. To accomplish this, we developed a feature called Autosegment. An annotator draws a box around an object and a segmentation mask is automatically generated. The annotator can then adjust the mask before marking it as complete. Under the hood, the subset of the image is sent through a model, which automatically identifies and segments the object of interest.<a href="https://scaleai.wistia.com/medias/lvnzhzfl7b" rel="noopener noreferrer" target="_blank">video</a>The Autosegment functionality was originally developed for autonomous vehicle data, which is a core customer use case at Scale. (You can read more about our <a href="https://scale.com/blog/autosegment" rel="noopener noreferrer" target="_blank">original Autosegment tool</a> in our previous blog post.) When it was first launched, we saw a 30% decrease in the time it took to produce segmentation masks. However, the use cases for Autosegment have expanded greatly since its introduction, with AV data now constituting only one of many domains for which Scale produces segmentation data. While the underlying model did generalize beyond the autonomous vehicle domain, we observed that it performed noticeably better on objects like street signs, vehicles, and pedestrians than on other types of classes.This article contains a technical deep dive into the machine learning model that powers Autosegment. We’ll walk through some of the choices we made during the development process as well as our recent effort to overhaul the model for improved generalizability. In particular, we will discuss the data-centric approach we took to boost the usefulness of Autosegment across a wider range of use cases.<h2>Model Anatomy</h2>The segmentation model developed to power this tool addresses a problem known as salient object detection wherein the goal is to identify and segment the most “salient” or prominent object in an image. There is subjectivity in this task because an image can easily contain more than one prominent item. The intention behind Autosegment is that the user draws an approximate bounding box around the object of interest. Based on this we can firmly make two assumptions:<ol><li>The target object will be relatively centered within the box</li><li>The target object will extend nearly to the edges of the box, with some nonzero padding</li></ol>We encode these assumptions into the training process. Specifically, images in the training set are cropped in such a way that the target object is always centered with some randomly added padding. For any training image containing multiple segmented objects, each object is cropped separately to produce an individual training observation. <img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fe23c12266e384e7ca60a35f93258ba3e">The architecture we chose to implement for this purpose is based on U2-Net proposed by <a href="https://arxiv.org/pdf/2005.09007.pdf" rel="noopener noreferrer" target="_blank">Qin et al.</a> for this type of problem. U2-Net provides two concrete benefits over other architectures we considered.First, it is designed to more effectively take into account both local and global contrast features of the image. Most of the popular feature extractors (VGG, ResNet, etc.) were designed and optimized for image classification. While they can work well for segmentation tasks, they do so by identifying features specific to the semantic classes they are trying to predict. The consequence of this is that they tend not to generalize as well to classes absent in the training set. On the other hand, U2-Net is designed to make use of image features that indicate object saliency. Because of this, while U2-Net may not perform as well at outright classification as more traditional architectures (we did not test this), in our testing it does a good job at identifying salient objects pertaining to obscure classes. This is important for Scale’s customers who often need to annotate data for niche use cases not represented in the training data.Second, we want to carefully avoid the need to stretch or downsample images. Because the goal is pixel-perfect accuracy, the model needs to output a high-resolution segmentation mask. Downsampling, stretching, or otherwise modifying the image in any information-reducing manner before feeding it to the model will typically result in blurrier segmentation masks than desired. To this end, utilizing a fully convolutional network allows for the processing of any image crop size or shape as long as it is large enough to be processed by the encoder phase of the U-Net (and small enough to fit within GPU memory). In addition, unlike some other architectures, U2-Net does not excessively downsample the input with convolution strides or pooling. This helps propagate enough information about the original image through the network to produce high-resolution mask outputs.<h2>Improving Generalizability</h2>Neural networks have a tendency to try and “cheat” by taking shortcuts when learning to solve a problem. Faced with the task of salient object detection, one shortcut is to simply learn how to segment the specific classes present in the training data. For example, if trained exclusively on autonomous vehicle camera data, the Autosegment model only needs to memorize what certain object classes look like, such as cars, trucks, and pedestrians. The resulting model may work for annotating vehicle scenes, but it will not perform well when applied to new object classes because it hasn’t truly learned the concept of “saliency.”<a href="https://scaleai.wistia.com/medias/dhzm8kaohk" rel="noopener noreferrer" target="_blank">Video</a>To overcome this challenge, the model needs to be given training examples for which this shortcut does not work. In the case of Autosegment, we expanded the training dataset to include hundreds of new object types. Neural networks have a finite number of parameters,&nbsp;meaning that there is a limit to how much “knowledge” they can hold. By providing more classes than the network can effectively memorize, the hope is that it will be forced to learn a simpler strategy that can detect saliency across a more general set of images.Our first step towards generalization was to incorporate the open-source <a href="https://cocodataset.org/#home" rel="noopener noreferrer" target="_blank">Common Objects In Context (COCO)</a> segmentation dataset into the training process. However, while COCO contains a wide variety of object classes, it only contains about 200 thousand annotated images, which is a small fraction of the training data from autonomous vehicle cameras. In addition, COCO segmentation masks are often somewhat imprecise.We then sought to leverage the vast segmentation data that has been produced using Rapid, our annotation platform for rapid experimentation. Rapid supports both full-scene segmentation and partial-scene segmentation. In full-scene semantic segmentation, every pixel in the image is classified into one of multiple classes. In partial-scene segmentation, pixels may be categorized into zero or more classes. Hundreds of thousands of images have been annotated using Rapid.<a href="https://scaleai.wistia.com/medias/15nwn1o6ff" rel="noopener noreferrer" target="_blank">Video</a>Full-scene segmentation often contains classes that are not beneficial to train on. For example, in outdoor scenes the sky is often a distinct class in the segmentation mask but is typically considered less salient than all other objects in the image. While we want to train on a diverse set of object classes, we do not want to train on “background classes” or objects that do not match a reasonable definition of saliency. Neural networks are generally robust to occasional adverse training samples, but in large quantities they can negatively affect training. In order to filter out these undesirable classes, we have taken the time to construct a mapping between a common taxonomy and the customer-specific classes. For example, this mapping helps us recognize that what one customer calls a “bicycle” is the same as what another customer calls a “bike.” From this, we filter out segmentations that do not match a list of approved classes. Partial-scene segmentation is the more common project type on Rapid and mostly avoids this problem of extraneous classes.<h2>Validation</h2>An important consideration for us was that because Autosegment is already in use across a large number of projects and use cases, changing the underlying model risks unknowingly worsening its accuracy on specific categories of images. In theory this type of accuracy loss could occur due to the finite information capacity of the model architecture. As mentioned earlier, the network needs to encode its problem-solving process using a limited set of parameters. If the number of parameters is too small, underfitting will occur and impede accuracy. We combat this possibility in two ways.First, we ensure that the training data is well-balanced. If the overall composition of the dataset is skewed in favor of any particular domain, then the model may tend to overemphasize training samples from that domain. Not only can this result in lower accuracy in other domains, but it could impede our goal of forcing the model to learn a generalized solution to the salient object detection problem.Second, in addition to benchmarking accuracy on the overall collection of images, we also benchmark the model’s performance on subdomains (such as autonomous vehicle data) to ensure it performs well within each of the strata. This essentially serves as “regression testing” for the model. Regression testing is a software development practice to ensure that code changes do not break existing known use cases. To accomplish this, we validate the model on independent test sets representing distinct customer use cases. The crucial use case we monitored during these improvements was autonomous vehicle data:<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fdb0470f15fa945cda7b946e690837b34">Beyond standard pixel classification metrics, we also calculate the precision and recall of pixels within a small neighborhood of the segmentation edges. These metrics help us assess how well this tooling will actually benefit annotators for whom the most time-consuming aspect of semantic segmentation is the object boundary pixels.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2Fac64ee3d7a814bc0852260ed4e5721f9">To calculate these edge metrics, we first start with the predicted and ground truth masks. Keeping only the single-pixel boundary of each mask, a <a href="https://en.wikipedia.org/wiki/Dilation_(morphology)" rel="noopener noreferrer" target="_blank">dilation operation</a> is applied with some small threshold. Metrics are then calculated from the dilated edges. The above image visualizes this process.<h2>The feedback loop</h2>Beyond the initial training of the model, data annotated with Scale Rapid continues to get fed back into the training process. This creates a feedback loop: Autosegment helps users to more quickly produce higher quality annotations, and in turn, those annotations serve to continuously improve the automatic segmentation accuracy. Having high-quality training annotations is an indispensable prerequisite to training a high-performing model. This <a href="https://scale.com/guides/data-labeling-annotation-guide" rel="noopener noreferrer" target="_blank">data annotation</a> lifecycle is a unique characteristic of Scale’s annotation platform and fundamental to how we achieve best-in-class annotation speed and accuracy.<img src="https://cdn.builder.io/api/v1/image/assets%2Fe0438815ba51486bbb6a202747122d4b%2F03a3cce06e0e4cfba0dd090356f2669d"><h2>How you can use Autosegment</h2>Autosegment is one of the multiple Active Tooling capabilities that exist in Scale’s products. Active Tools provide interactive ML-powered assistance to data annotators. By letting humans make changes to the model inputs in real time, model outputs can be quickly dialed in to match the situation precisely and minimize time spent on the task.You can supercharge your data annotation using <a href="https://scale.com/studio" rel="noopener noreferrer" target="_blank">Scale Studio</a>, which offers best-in-class annotation infrastructure to accelerate your in-house annotation team. Studio also offers a free tier to familiarize yourself with the platform and start annotating data. If you’ve decided you’d like to offload more parts of the data annotation pipeline, <a href="https://scale.com/rapid" rel="noopener noreferrer" target="_blank">Scale Rapid</a> provides an easy way to send your data to be annotated by Scale and get production-quality labels with no minimums.

How Scale AI’s computer vision annotation tooling (CVAT) uses machine learning-assisted tooling for faster and more accurate semantic segmentation.

Spencer Papay

How Much Better is OpenAI’s Newest GPT-3 Model?

Machine Learning Assisted Image Segmentation