Skip to main content

​
Setting up optimization

Before you can optimize an agent, you’ll need to have an evaluation set up. This lets you get more granular about success/failure cases of your model, and lets us feed this feedback into the training process.

​
Creating an evaluation

To create an evaluation, navigate to the Evaluations tab on the left sidebar and click the New Eval button on the top right.
You want to be as explicit as possible about important success/failure cases of your model, in order to let Vogent know how to judge a call. We can also automatically generate eval criteria based on the talk track and recordings you upload, though we recommend you review the generated criteria and make sure it aligns with your use case.

​
Optimizing the agent

Once you’ve created an evaluation, you can optimize the agent by clicking the “Optimize” button next to the prompt.

​
Select dials

You should select the dials you want to use as an optimization target. We evaluate the performance of the model on those historical dials, and use that feedback to train the model. We will generally keep the prompt the same, and only train the model weights, so you should make sure there aren’t any obvious oversights in the prompts that may be causing the issues, or explicitly indicate you want us to optimize the prompt, and not the model weights.
You should select a dataset of approximately 100 dials.

​
Review the optimized model

The model will automatically evaluate the version you selected and train on the selected dials. The new version will be created in under the agent, and you can use our standard tools to test the version and evaluate the performance, like Counterfactuals and Evaluation.