Computer scientist Roger Craig built an app to prepare for Jeopardy domination.

http://thenextweb.com/shareables/2011/11/16/mind-blown-this-guy-broke-jeopardys-all-time-record-with-an-app/

by soy714 • 15 years ago

157 24 15 years ago

Most recent tournament champion, Roger Craig, broke Ken Jenning's all-time money record.<p>His finest display of Jeopardy prowess was this back-to-back "Daily Double" all-ins. http://www.youtube.com/watch?v=YRwK8SyVeJE<p>To prepare for the competition Craig developed a web app to analyze all the questions in Jeopardy's archive and reverse engineer the game. He explains it here. http://vimeo.com/29001512

klochner• 15 years ago

I'd like to see Roger go up against Watson, since both were "trained" for Jeopardy

loumf• 15 years ago

One advantage Roger has is insane betting (which could backfire, obviously). I didn't watch Watson -- did it bet so aggressively on Daily Double questions?

ssharp• 15 years ago

Watson was able to analyze the current situation and make a rational bet on Daily Doubles. It analyzed it's comfort in the category the Daily Double was in, and then used a regression model to pick the optimal bet size, which factored in how winning or losing the Daily Double would change Watson's chances of winning or losing the game.

I'm not really sure if Watson's model goes as far to consider other players' Daily Double behavior, or simply examines other players' scores. It would make sense to try and look at the game theory aspect though, especially in an iterative-game scenario. If a player bets hyper-aggressively on Daily Doubles (and Final Jeopardy) and gets most of them right, you'd think Watson would want to take that into consideration.

nonsequ2• 15 years ago

My bet is that he'd lose. Beyond a certain level of knowledge, successful Jeopardy is all about hitting the buzzer first. Watson has a huge advantage in that respect because it gets the signal to buzz in electrically.

johno215• 15 years ago

This.

The Watson show was a not a completely fair match-up since Watson had a much better average buzzer time than his human opponents.

If this was not the case, I would speculate a really good human opponent could beat Watson at least some of the time.

eru• 15 years ago

Actually, it should be harder for Watson to hit. Since Watson waits for the end of the question. A human can anticipate that.

mukyu• 15 years ago

Watson had a physical buzzer pusher. While he may have an advantage of reflexes, he did not do it fully electronically (obviously he 'electronically' activated the motor that pushed the button).

MichaelApproved• 15 years ago

My method to eliminate the buzzer issue:

Allow both contenders to press the button within the first 2 seconds and, if they both buzz in within the 2 second window, the human answers first, because Watson could be trusted to "lock-in" it's answer.

Points are issued to whomever gets the answer right, allowing for both to receive or lose points on the same question.

zitterbewegung• 15 years ago

The only thing I can get from this article is that he created a study tool that wasn't a set of papers or a document but actually a computer program. I don't see how this has any significance other than he created a computer program. I wish there was more information to this article other than fluff.

mpanzarino• 15 years ago

The information is in the second video, I figured, why transcript Craig's description when he did a succinct job of it himself?

To be honest, there's little information other than what he divulged out there on the net, as it was a private tool. If I find more i'll write a followup, it's pretty fascinating.

soy714• 15 years ago

There's a video of Roger Craig explaining the web app he created: http://vimeo.com/29001512

DanielRibeiro• 15 years ago

The wikipedia article contains a lot of detailed info on how well he did: http://en.wikipedia.org/wiki/Roger_Craig_%28Jeopardy!_contes...

danteembermage• 15 years ago

I think the big innovation highlighted in the video is that he used historical data on frequency of certain topics, the dollar value assigned to those topics, and his personal performance on certain categories to figure out which categories were most important to study. I know that I am bad at fashion and 19th century French literature but what relative fraction should I spend on one over the other? His code gave him a number and that's the ratio he studied them. So you're right, at that point it's not much different from flipping through a giant study manual, other than the manual telling you should should really look at pages 3, 11, 21, 47, 96, ... because that's what the solution to the nonlinear optimization problem said would give you the best chance at winning against simulated opponents.

zach2• 15 years ago

The amount of Jeopardy! questions that are online in J!-Archive is by now quite large. So large that it has begun to approach the corpus of facts from which knowledge-based non-current-events Jeopardy! questions are drawn.

Although Jeopardy does not repeat questions per se, they often find different ways to clue the same fact or connection.

I would guess that if you took the text of a clue, ran some trivial clustering on the J!-Archive contents, and instantly displayed the top three clues and responses for a human, they could be quite competitive on the show even if they knew nothing on their own.

So if you put the J!-Archive into a Leitner-system flashcard app and put a few hundred hours into it, I imagine it would improve a contestant's chances considerably. It doesn't look like much, but that looks like the major tool here. If I had won my last game (the final regular game of Season 23, followed by a six-week hiatus) I was planning something like this. However, I never thought of using analytic feedback with categorization in this way. That clearly also gave him some valuable feedback on what to study and what categories to be confident about.

However, this is to take nothing away from Roger Craig's exceedingly impressive untrained skill at the game, which he was able to train effectively to unreal levels. He plays Jeopardy! like a boss.

zach2• 15 years ago

Wow. This is a far more thorough approach than the experience-focused app I made in 2007, but then again Roger Craig is also a lot better at Jeopardy! than I am.

http://zachbaker.com/how-to-win-on-jeopardy-with-ruby-on-rai...

georgieporgie• 15 years ago

I somehow missed the data-scraping introduction, that he sourced a fan-operated website that archives questions and answers. Here I'd thought he'd just off-handedly created a phenomenal data scraper for the general web.

He didn't talk much about the algorithm he used to present questions for optimal memorization. I imagine it was spaced repetition, as used in SuperMemo:

http://www.wired.com/medtech/health/magazine/16-05/ff_woznia...

gwern• 15 years ago

Yes, that's right. That was my own question; he said he uses Anki: http://quantifiedself.com/2011/09/roger-craig-on-knowledge-t...

mpobrien• 15 years ago

A while back I actually built a mobile site for playing Jeopardy (originally conceived as a drinking game based on jeopardy, but that's another story...) part of which was a scraper that pulled data off the fan-sourced site and parsed it out.

Code for parsing, if anyone's interested: https://github.com/mpobrien/jeoparty/blob/master/scrape_mong...

And code for scraping: https://github.com/mpobrien/jeoparty/blob/master/curler.sh

adaml_623• 15 years ago

"Of course, if he’s smart enough to build a focused learning app that allowed him to dominate the show like this, it’s probably likely he would have done ok on his own."

The author of the article just doesn't get it.

Computer scientist Roger Craig built an app to prepare for Jeopardy domination.

Related Stories