In this section you can view, add, and update models!

To get started with adding a model, click + Add Model in the top right corner. If you want to modify an existing model, click the model name, then click Modify Model.

When adding or updating a model you can specify whether to include a public api endpoint that we use to generate responses, or upload a set of pre-generated responses that map to a prompt set. A model can include both the endpoint or uploaded responses, but only one is necessary. Upon eval run creation you can specify whether to use the endpoint or uploaded response set(s) for each model depending on which the model is configured to use!

1. Set up Model Endpoint - Here you can add in the model endpoint and any necessary parameters. You can also test the endpoint to make sure it's been set up correctly.
2. Upload Model Responses
   1. Here you will first be prompted to select a prompt set that you plan to upload responses for.
   2. Once a prompt set has been selected, upload a csv that contains the model responses. The csv needs to contain a reference_id and response column, where the reference_id maps to the reference_id in the prompt set. The reference_ids will need to correctly map to each prompt in the prompt set in a 1 to 1 manner, otherwise the upload will throw an error (each prompt in the prompt set must have a response).
      
   3. After you have finished uploading, you'll see "Upload Success! Continue by clicking finish" at which point you can click Finish to finish adding or updating your model.
      

Return to: Model Evaluation