Major Mixed Integer Linear Programming Solver Faked Performance Data

http://www.gurobi.com/company/news/announcement

by whatever1 • 8 years ago

14 4 8 years ago

This title seems like an exaggeration of what is claimed in the article. In the article, they state that they benchmarked their solver in a biased way that made their solver look like it performed better than it did, not that they faked performance data altogether.

whatever1• 8 years ago

Far from exaggeration. In fact CPLEX and Express decided to go nuclear and remove their solvers from the benchmark set, so now we will be in the dark, regarding relative performance http://plato.asu.edu/bench.html

shaklee3• 8 years ago

This is pretty sad. Unlike databases and other software, MIPS solvers have very good benchmarking standards, and though they don't cover every case, you can typically see which solvers are best for certain types of problems. Having met the Gurobi founders, they genuinely don't seem like they'd do something like this on purpose.

Now Mitellman's only benchmarks for this category will be the slower solvers.

leethargo• 8 years ago

I guess there will still be third-party benchmarks, in particular as part of the papers that go along the updates to the MIPLIB sets of benchmark instances?

Major Mixed Integer Linear Programming Solver Faked Performance Data

Related Stories