That it is 90% cleaning, and only 10% modeling. Never had a dataset that was ready to use like the ones on Kaggle. Most of the time I get a mixture of Excel Sheets with weird formatting, .csv files and SQL dumps that have questionably encoding, and lots of unnecessary information and missing values.
What would be the best data structure for using this in, say, a Python script? I imagine just putting it into a dictionary (hash map etc.) won't work because of the size.