I added context data to the TruthfulQA dataset

https://huggingface.co/datasets/portkey/truthful_qa_context

by roh26it • 2 years ago

1 1 2 years ago

roh26it• 2 years ago

Being one of the most downloaded datasets on Huggingface, I was a little bit surprised by how dirty this dataset was. Plus it had very limited information and some incorrect classifications as well.

For an internal experiment on building a "Truthful Evaluator", we picked up this dataset and tried fine-tuning a model on these 8000 odd examples.

Realised that it needed: 1. Cleaning up 2. Some reclassification

But, most importantly - it lacked context data. It only had a link pointing to the source which was also absent for a few rows.

We scraped the internet for the link in the dataset, matched it to the question and narrowed down on a small context to be added to the main dataset.

Releasing it publicly so that someone else may avoid the 2-3 days of pain of wrangling with this data.

I added context data to the TruthfulQA dataset

Related Stories