ox/
CommaQA
Datasets
Models
Docs
Blog
Pricing
Search
DataBranchesEvaluationsNotebooksFine tune
Evaluations
Run models against your data
Introducing Evaluations, a powerful feature designed to enable you to effortlessly test and compare a selection of AI models against your datasets.
Whether you're fine-tuning models or evaluating performance metrics, Oxen evaluations simplifies the process, allowing you to quickly and easily run prompts through an entire dataset.
Once you're happy with the results, output the resulting dataset to a new file, another branch, or directly as a new commit.
419ae91b-d9ce-463d-afff-c724d151eb9e
419ae91b-d9ce-463d-afff-c724d151eb9e
OpenAIOpenAI/o1 Minitext → text
Ox
ox
8 months ago
answer the question

{question}
started (20%)Running 1 / 5 row sample732 tokens$ 0.0000 1 iteration
29ed719b-182c-4a56-a973-56a8966e70c8
29ed719b-182c-4a56-a973-56a8966e70c8
Mistral AIMistral AI/mistral-small-2409text → text
Ox
ox
8 months ago
Answer the question

{question}
main-testing
qa_test.parquet
completed 100 rows23504 tokens$ 0.0141 1 iteration
Testing mini mistral
7540e775-a751-41f9-99cc-7ca1129fc1df
Mistral AIMistral AI/ministral-3b-latesttext → text
Ox
ox
8 months ago
Answer the following question given the context. Answer in a comma separated list.

For example
Question: What movies have people from the country Whime acted in?
Context: movie: Geissant ; directed by: Carpoon. movie: Myristorrhoid ; directed by: Carpoon. movie: Riften ; directed by: Muntaril. movie: Chimpwurst ; director: Midcareer. movie: Chimpwurst ; award: Neuropsychotaxis. movie: Riften ; awarded: Neuropsychotaxis. movie: Geissant ; awarded: Triptychology. movie: Myristorrhoid ; award: Monoxandrite. award: Lameze ; winner: Carpoon. award: Lameze ; winner: Midcareer. award: Modiparity ; winner: Muntaril. movie: Myristorrhoid ; release year: 1974. movie: Chimpwurst ; year: 1962. movie: Geissant ; year: 2019. movie: Riften ; year: 1974. Midcareer acted in the movie Myristorrhoid. Carpoon was an actor in the movie Chimpwurst. Muntaril acted in the movie Geissant. Carpoon was an actor in the movie Geissant. Midcareer acted in the movie Chimpwurst. Muntaril acted in the movie Riften. Midcareer was born in 1943. Carpoon was born in 1921. Muntaril was born in 1921. Midcareer is from the country of Whime. Muntaril grew up in the nation of Whime. Carpoon grew up in the nation of Poquet. Midcareer was one of the producers of the movie Riften. Carpoon was one of the producers of the movie Geissant. Carpoon produced the movie Riften with others. Muntaril produced the movie Chimpwurst with others. Carpoon was one of the producers of the movie Myristorrhoid. Midcareer was one of the producers of the movie Myristorrhoid. Muntaril produced the movie Geissant with others. Muntaril wrote for the movie Geissant. Carpoon was one of the writers for the movie Chimpwurst. Midcareer was one of the writers for the movie Myristorrhoid. Muntaril wrote for the movie Riften. Midcareer was one of the writers for the movie Geissant.
Answer: Myristorrhoid, Chimpwurst, Geissant, Riften

Question: {question}
Context: {passage}
Answer:
started (68%)Running 68 / 100 rows74752 tokens$ 0.0026 2 iterations
9d1e8ce6-12d0-4a58-8739-5a453dd45d5c
9d1e8ce6-12d0-4a58-8739-5a453dd45d5c
Mistral AIMistral AI/mistral-nemotext → text
Ox
ox
8 months ago
answer the question

{question}
started Running Waiting... 0 tokens$ 0.0000 1 iteration
running
3430db2e-4bc1-4e0f-a285-6b533841ca43
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
{passage}

{question}
main-bad-results
qa_test.parquet
completed 100 rows57042 tokens 2 iterations
eval name
7fee4a15-da34-428c-978e-c8ee4f60e15a
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
{passage}

{question}
test-one-shot
qa_train.parquet
completed 300 rows170472 tokens 2 iterations
LLM As A Judge
df8baca0-3e33-47d2-8982-95614a499ffd
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
Are these two lists the same? Answer with true or false, one word, all lowercase.

List 1: {answer_spans}
List 2: {prediction}
test-one-shot
qa_test.parquet
test-one-shot
qa_test.parquet
completed 100 rows5041 tokens 3 iterations
Evaluation
707ba35a-9cb7-4c75-9029-477a1cbdbed6
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
Answer the following question given the context. Return the answer in a comma separated list. For example:

Question: What movies have people from the country Whime acted in?
Context: movie: Geissant ; directed by: Carpoon. movie: Myristorrhoid ; directed by: Carpoon. movie: Riften ; directed by: Muntaril. movie: Chimpwurst ; director: Midcareer. movie: Chimpwurst ; award: Neuropsychotaxis. movie: Riften ; awarded: Neuropsychotaxis. movie: Geissant ; awarded: Triptychology. movie: Myristorrhoid ; award: Monoxandrite. award: Lameze ; winner: Carpoon. award: Lameze ; winner: Midcareer. award: Modiparity ; winner: Muntaril. movie: Myristorrhoid ; release year: 1974. movie: Chimpwurst ; year: 1962. movie: Geissant ; year: 2019. movie: Riften ; year: 1974. Midcareer acted in the movie Myristorrhoid. Carpoon was an actor in the movie Chimpwurst. Muntaril acted in the movie Geissant. Carpoon was an actor in the movie Geissant. Midcareer acted in the movie Chimpwurst. Muntaril acted in the movie Riften. Midcareer was born in 1943. Carpoon was born in 1921. Muntaril was born in 1921. Midcareer is from the country of Whime. Muntaril grew up in the nation of Whime. Carpoon grew up in the nation of Poquet. Midcareer was one of the producers of the movie Riften. Carpoon was one of the producers of the movie Geissant. Carpoon produced the movie Riften with others. Muntaril produced the movie Chimpwurst with others. Carpoon was one of the producers of the movie Myristorrhoid. Midcareer was one of the producers of the movie Myristorrhoid. Muntaril produced the movie Geissant with others. Muntaril wrote for the movie Geissant. Carpoon was one of the writers for the movie Chimpwurst. Midcareer was one of the writers for the movie Myristorrhoid. Muntaril wrote for the movie Riften. Midcareer was one of the writers for the movie Geissant. 
Answer: Myristorrhoid, Chimpwurst, Geissant, Riften 

Question: {question} 
Context: {passage} 
Answer:
test-one-shot
qa_test.parquet
completed 100 rows104817 tokens 3 iterations
llm as a judge
611ab188-91bb-4f0e-8063-79d31bb0b285
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
are these lists equivalent? answer with one word "true" or "false" all lowercase

List 1: {answer_spans}
List 2: {prediction}
one-shot-results
qa_train.parquet
one-shot-results
qa_train.parquet
completed 300 rows15245 tokens 3 iterations
Question Answering
f2a3ce71-2ed2-42b8-aa37-9dc6d3d991e7
OpenAIOpenAI/GPT-4otext → text
Ox
ox
8 months ago
Answer the following question given the context. Answer in a comma separated list.

For example
Question: What movies have people from the country Whime acted in?
Context: movie: Geissant ; directed by: Carpoon. movie: Myristorrhoid ; directed by: Carpoon. movie: Riften ; directed by: Muntaril. movie: Chimpwurst ; director: Midcareer. movie: Chimpwurst ; award: Neuropsychotaxis. movie: Riften ; awarded: Neuropsychotaxis. movie: Geissant ; awarded: Triptychology. movie: Myristorrhoid ; award: Monoxandrite. award: Lameze ; winner: Carpoon. award: Lameze ; winner: Midcareer. award: Modiparity ; winner: Muntaril. movie: Myristorrhoid ; release year: 1974. movie: Chimpwurst ; year: 1962. movie: Geissant ; year: 2019. movie: Riften ; year: 1974. Midcareer acted in the movie Myristorrhoid. Carpoon was an actor in the movie Chimpwurst. Muntaril acted in the movie Geissant. Carpoon was an actor in the movie Geissant. Midcareer acted in the movie Chimpwurst. Muntaril acted in the movie Riften. Midcareer was born in 1943. Carpoon was born in 1921. Muntaril was born in 1921. Midcareer is from the country of Whime. Muntaril grew up in the nation of Whime. Carpoon grew up in the nation of Poquet. Midcareer was one of the producers of the movie Riften. Carpoon was one of the producers of the movie Geissant. Carpoon produced the movie Riften with others. Muntaril produced the movie Chimpwurst with others. Carpoon was one of the producers of the movie Myristorrhoid. Midcareer was one of the producers of the movie Myristorrhoid. Muntaril produced the movie Geissant with others. Muntaril wrote for the movie Geissant. Carpoon was one of the writers for the movie Chimpwurst. Midcareer was one of the writers for the movie Myristorrhoid. Muntaril wrote for the movie Riften. Midcareer was one of the writers for the movie Geissant.
Answer: Myristorrhoid, Chimpwurst, Geissant, Riften

Question: {question}
Context: {passage}
Answer:
one-shot-results
qa_train.parquet
completed 300 rows313614 tokens 3 iterations