Using Model Tuning to Beat Vegas

http://blog.sigopt.com/post/136340340198/sigopt-for-ml-using-model-tuning-to-beat-vegas

by Zephyr314 • 11 years ago

3 7 11 years ago

blog.sigopt.com

Zephyr3144• 11 years ago

Hello, I'm the author of this post and co-founder of SigOpt (YC W15). Let me know if you have any questions about the post or what we do at SigOpt.

All of the code used in this post can be found at https://github.com/sigopt/sigopt-examples

Oxydepth• 11 years ago

Very in depth. Good read.

Zephyr3144• 11 years ago

Thanks! Let me know if you have any questions.

mswen2• 11 years ago

Zephyer314 hello, interesting read. It is interesting to me that the lowest cumulative level for the tuned model is approximately equal to the lowest cumulative level for the simple model with no tuning. In fact if we had been only looking at the results up until Dec. 12 or so we would have concluded that the simpler model works better and barely beats the house and that maybe the performance of the simple and tuned models are converging.

However, they now seem to be diverging with a clear advantage to the tuned model. If you had been actually using this to bet you might have given up on the tuned model around Dec. 7th when draw down was at its worst.

What will really be interesting is whether the performance over the rest of the season continues to diverge in the current direction or if the performance tends float up and down around the break even line.

Is one of the parameters a progress through the season index? Anyway thanks for sharing the example.

Zephyr3144• 11 years ago

Thanks! Great question.

There is a huge variance in the cumulative gains as you pointed out. Because of the 100 win/110 loss edge that Vegas has you need to win over 52.5% of the time to register a profit. Even with a model that can win 52.5% + e of the time you are effectively flipping a (biased) coin. Over the 131 games in the holdout dataset this random walk will tend upward, but can vary widely (especially at the start). As the season continues on this should start to wash out (in the cumulative profit). We plan to write a followup post at the end of the season to see how it performs. You could also fork the code [1] and train/tune on '00-'13/'13-'14 and use the entire '14-'15 set as the holdout (or a similar combination).

The features can be found here [2], we don't use individual player stats or temporal stats, but it should be relatively easy to add if you wanted to try it out.

[1] https://github.com/sigopt/sigopt-examples

[2] https://github.com/sigopt/sigopt-examples/blob/master/sigopt...

mswen2• 11 years ago

Thanks for the reply. I don't know if I have sufficient interest to actually grab the code and try it out. However, I will try to remember to look for your update post at the end of the season.

Using Model Tuning to Beat Vegas

Related Stories