Debunking a Computer Chess Scandal

http://www.chessbase.com/newsdetail.asp?newsid=7791

by worldvoyageur • 15 years ago

77 38 15 years ago

Simply on the basis of telling a fascinating story, the series is well worth reading. Beyond that, some of what particularly struck me was that:

- since about 2005, chess programs that run on a regular PC have consistently outperformed the best human grandmasters, and the skill of the computer chess programs continues to improve. 2005 was a quantum shift in the chess programming world, when chess programs first played "brilliant games with deep, beautiful combinations...[and] routinely produce[d] highly artistic masterpieces of chess while avoiding a great many pointless “computer” moves that for many years had been a source of ridicule among strong human players."

- improvement in algorithms is driven by an active community ecosystem, open source contributions and flagrant copying of the brilliant insights of a few. No matter how brilliant the insight, the community is able to improve upon it. Intellectual property protection would have stopped much of this improvement in its tracks.

It's four separate articles, so I posted the first. All four are:

http://www.chessbase.com/newsdetail.asp?newsid=7791 (part 1)

http://www.chessbase.com/newsdetail.asp?newsid=7807 (part 2)

http://www.chessbase.com/newsdetail.asp?newsid=7811 (part 3)

http://www.chessbase.com/newsdetail.asp?newsid=7813 (part 4)

wisty2• 15 years ago

A quote from Rajlich:

> Yes, the publication of Fruit 2.1 was huge. Look at how many engines took a massive jump in its wake: Rybka, HIARCS, Fritz, Zappa, Spike, List, and so on. I went through the Fruit 2.1 source code forwards and backwards and took many things.

It's not as if he never admitted to studying Fruit.

Also, most of the confusion is there because Rajlich didn't have a repo until Rybka 4, so when the reverse engineered his code they formatted it the same way as Fruit, making it look very similar. If a judge missed the bit saying that the Rybka code is "functionally equivalent code", not the actual code, they might be fooled into thinking the orignal codebases looked very similar. The guy doing this wrote something like 30 posts a day on the discussion board about Rybka, and was clearly not a fan.

They should have made a 3-way comparison - Rybka's PST, Fruit's PST, and some independent program's PST (in the same format as the other two).

jules• 15 years ago

What is a good resource to learn about the important advances in computer chess?

Bootvis• 15 years ago

A response has come from the author and panel members on Chessvibes, see the pdf links in here:

http://chessvibes.com/reports/controversy-over-rybkas-disqua...

copper• 15 years ago

For what it's worth, the WCCC's Rule 2 seems to be a lot like a university honor code (if you do borrow code or ideas, make sure you cite it.) I don't believe that's too hard to follow. Maybe someone should fund a kaggle competition to design the best chess-playing program, with the final requirement of making the code open-source after :)

Given that most of what you'd implement a computer chess engine is pretty much available online (at most,behind a paywall), and digging through the source code of GNUChess and Crafty would give a lot of insight into the scoring function, so, yes, the score function would be pretty much similar in most parts unless someone comes up with a radically new way of doing things.

wbhart• 15 years ago

From the article, "By definition, plagiarism only happens when credit to sources is not given, which was never the case with Rybka."

I simply do not agree with this. I cannot copy slabs of open source code, incorporate them into my product and pass if off as my own just because I somewhere credited the great performance of my program to stuff I learned from reading open source software.

To me, plagiarism is copying some portion of someone else's work and including it in a work which I (explicitly or implicitly) claim to be my own.

For example, a student handing in a wikipedia article for a homework assignment still commits plagiarism even if they credit wikipedia as a source but don't explicitly say the thing is a wikipedia article. They hand it in as their work when they did not write it. So at this point I do not feel I agree with the article. It's as though the argument is, yes he copied, but it's ok because he gave "credit", there were plenty of new ideas and everyone else was doing it too. That's not the same thing as, "he didn't commit plagiarism". The latter means he did not copy the code.

Having said that, I am absolutely gobsmacked that the committee did not simply ask for the source code to both programs, check that it in fact compiles to the binaries in question, then do a comparison for legally significant quantities of identical code.

It's even more remarkable when you realise that the version of Fruit involved was open source, so they didn't even need to ask for the source code to that!

Note: I edited the above in response to the comments below.

sp332• 15 years ago

I disagree. I think if a student turns in an assignment that's clearly and admittedly just a WP article, the student should not get credit for doing the work, but he didn't commit plagiarism.

SeanLuke• 15 years ago

I don't have an opinion on what this guy did and whether he should have been stripped, but you're redefining plagiarism pure and simple.

Plagiarism is passing off someone else's ideas or expression of those ideas as if they were your own. And that's it. Plagiarism has nothing to do with whether you benefit from someone else's ideas illegally or unethically.

Let's say that there's a competition to write an original novel. I take Moby Dick and change one sentence. Then I submit it with a big warning on the front cover that says: "This novel is Moby Dick with one sentence changed by me." I have violated the rules of the contest perhaps. If I win I maybe benefitted from it. But I have not committed plagiarism -- I did not pass off these ideas as if they were my own. It was made clear whose ideas they were.

jsnell6• 15 years ago

What makes you think Rajlich would have given them source access, or that it wasn't asked for? By all accounts he was not willing to assist with the investigation in any way.

crististm• 15 years ago

I've read the report files. They based the findings on the behaviour of the Rybka binary. As the article says, Rajlich refused to show the source code.

bambax• 15 years ago

> Having said that, I am absolutely gobsmacked that the committee did not simply ask for the source code to both programs

From the article (part 4):

> Critics of the ICGA soon realized that no one has actual Rybka source code from before 2010, not even Rajlich himself, who sheepishly admitted to Nelson Hernandez off-camera in the course of their July 2011 video interview that he had never maintained any form of version control for Rybka source code until Rybka 4.

mark_l_watson• 15 years ago

Good article. I agree that to slam someone's reputation in public, the ICGA should have had much better evidence. My understanding is that the release of Fruit source code inspired many chess programmers.

medusa666• 15 years ago

It has been discussed (in chess circles) in excruciating and unambiguous detail how Rajlich blatantly ripped off (Google it) open source code, thereby violating tournament rules, open source licenses, etc. This "debunking" series of articles by Chessbase is a pathetic attempt to reinvent the past on behalf of one of their most lucrative products, and perhaps to fool a wider audience who didn't follow the original scandal when it happened.

I used to be a big fan of Rybka, Rajlich and Chessbase, but these continued denials just make a bad thing worse. It's like Floyd Landis, Tyler Hamilton, and the third guy.

wisty2• 15 years ago

The thing I'd like to see - a three way comparison of the alleged "cheating". Rybka and Fruit have some similar parts, but how about Fruit and any other similar program? It might just be that any chess program will have parts that look similar.

jsnell6• 15 years ago

A document like that was included in the original investigation, with 9 different chess engines. It didn't compare source code though, but on which features of the board each engine would consider when evaluating a position. For example one engine might give an extra boost to a position where your knight is in a central location and capable of affecting a larger part of the board, another might give a boost to situations with interlocking pawn defenses, etc.

According to the evidence, two engines with different ancestry would have a feature overlap of about 30-35%, up to 45% in the most extreme cases. With the exception of Fruit and Rybka, with 75% overlap.

Almaviva• 15 years ago

As far as I can tell, the basic facts are:

1. There is damning evidence (probably beyond reasonable doubt) that Rybka contains some copy-pasted code from Fruit (the open source program).

2. The interesting thing about Rybka was that it was dramatically better than any chess engine of its time, due to original improvements.

3. Many people are bitter that Rybka became an extremely successful commercial product, due to being clearly the best chess engine of its time. It also made several formerly viable commercial products non-viable due to being much better.

4. A mass of clones obtained by either disassembly or obtaining source code appeared, which are now equal or slightly superior to Rybka, and this has essentially destroyed its commercial viability.

For people who value the letter of the law, #1 is damning, and Rybka cheated, period. This isn't beyond theoretical doubt, but it's beyond a reasonable one.

But, given that Rajlich could have easily and legally re-implemented the same algorithms in the mudane parts of Fruit that are in question without exact code reuse, and that what is significant about Rybka is actually the original and innovative part, there's a pretty good argument that he's well within his rights to profit from his creation.

waqf2• 15 years ago

Which did you see as the conclusive evidence that code was copy-pasted? Section 4 of the article made the point that the Rybka source was missing and the accusers' evidence was decompiled code which they had arranged to look as similar as possible.

harryh• 15 years ago

Are there some good links I can read about claim #1? I have only just been introduced to this story via the articles on ChessBase and I would like to read "the other side of the story" so to speak.

ww520• 15 years ago

OT: why do websites find it useful to disable the up/down arrow keys? It is not. It's very annoying. Make navigation a pain in the butt. Make people never want to come back to your site.

jmmcd• 15 years ago

Maybe it's the fault of the game-history widgets on the page. They use left and right arrow keys for game navigation, so it's possible they steal up and down arrows also.

mml• 15 years ago

A fine example of terrible writing.

rwmj• 15 years ago

For UK/Hungary computer nerds, Dr David Levy was also responsible for the Enterprise 64 computer: https://en.wikipedia.org/wiki/Enterprise_%28computer%29

bjornsing• 15 years ago

Isn't there something slightly ironic about accusing the best pupil in the class of peeking over the shoulders of the other students during tests? Not that it can't be a correct accusation, but if this guy "cruised to victory in four consecutive WCCC tournaments in 2007, 2008, 2009 and 2010" then he must have done something right (other than copy-paste).

From a copyright perspective I can certainly see an issue, but that would be between Rajlich and the author of this Fruit I would say.

dalke6• 15 years ago

Actually, the argument isn't about cut&paste programming. That would be somewhat easy to determine by simply comparing the binary output (after some searching for the right compiler options from the compiler for that era).

Indeed, the accusations clearly say that there were significant changes to the code, which makes a reverse engineered comparison harder to do.

Instead, the complaint is that the underlying algorithms were re-implemented, instead of being original algorithms.

janzer• 15 years ago

Although the articles seem to not mention it, it should probably be noted the author is a moderator on the Rybka forums so is most likely not coming from an exactly unbiased viewpoint.

worldvoyageur2• 15 years ago

Though buried as an aside in the fourth part of the piece, the author does mention that he is the moderator on the Rybka forums.

"In my capacity as Rybka forum moderator I have access to posting statistics."

Indeed, his moderator status leads to one of his core debunking arguments. Some Dr. Hyatt guy, apparently a key individual behind the accusations against Rybka, appeared to be making the attack a full-time job, shifting the arguments as needed to defend a guilty conclusion.

Though he doesn't say it, I suspect that noticing this was impetus for the author to seriously tackle a look at the charges from the defense perspective.

Not that there is anything wrong with either side picking a conclusion (guilty, not guilty) and then working hard to support it. However, part of the credence given the initial charge is that it resulted from an independent search for truth. This does not appear to have been the case. Instead, the charges were the prosecution half of an adversarial process. In these articles, now we see the defense.

Caveat: all I know about this matter is what I read in the four part article posted here.

Debunking a Computer Chess Scandal

Related Stories