Why I hate programming competitions

http://web.archive.org/web/20071121053436/http://www.cs.caltech.edu/~mvanier/hacking/rants/programming_competitions.html

by gnosis • 17 years ago

62 25 17 years ago

web.archive.org

_pi• 17 years ago

I've been to several highschool/college level competitions they generally fall into two types, writing scenario code the fastest with no concern for real-world problems, or a test in reading obfuscated code (GE/Fairfield University you're seriously incompetent in your inability to provide actual computers.). In my experience the later is worse, much worse, more-so because its frustrating as hell.

loup-vaillant• 17 years ago

I did these, too. The goal wasn't the podium, though. It was graduation.

martincmartin• 17 years ago

One problem I've seen with people who do really really well in programming competitions is that they can ace the coding part of an interview, even if they're not very good. I've worked with two different people who did really well at programming competitions. One was a good hire, the other wrote lots and lots of code that only kinda worked.

gms• 17 years ago

Do you have any proposals for how to fix this from the interview's point of view?

jacquesm• 17 years ago

Interviews don't work to ascertain how well someone will work as a programmer other than the most trivial reject / accept scenarios. Everything else plays out in the first two weeks to a month after hiring.

Interviews select for people that are good at doing interviews, they present themselves well and may be able to subconsciously flatter the person interviewing them.

Because of that, if the number of applicants for a given job is high the interviewer has an easier job of it, he/she can simply raise the bar during the interviews, eventually someone will pass the bar and that person will get hired.

The reverse, in a market where talent is scarce the bar will be lowered until someone gets hired.

But all that says is that during the interviewing phase the person performed 'as expected'.

How well they work in a team, what their work attitude in general is and so on is still largely unknown. The same goes for the quality of their production.

Interviewing is a 'rough' selection process, you try to do a sort of 'triage' here.

The groups are hire, wait-and-see, and no way.

The 'wait-and-see' might not get the nod this round but in a next round when quality is scarcer they might come up for a second round. There are probably plenty of people in this second group that would outperform people in the first group. They're just not that good at 'selling' themselves.

derefr• 17 years ago

Make the coding part of a larger flow. Instead of saying "there's something wrong with this code; find it and fix it," perhaps say "this is a (synthetic) bug database, and this is a repository for the project it refers to. Prioritize and fix the bugs you think are the most important. Don't worry about not knowing the codebase—I want to see how you go about learning it." Give them a day or two (and make sure there's way more than two days' worth of work entered as bugs); let them work from home, and track their commits.

kqr2• 17 years ago

Interviews are more like sprints whereas the actual job is like a marathon.

icefox2• 17 years ago

How about a competition where:

  Round 1) You write some core algorithm
  Round 2) You write a library that uses algorithm
  Round 3) You write a tool that uses the library
  Round 4) You write a different tool that uses the library, but have 1/2 the time

  The end tools would be tested with a million edge cases.

  Round 5) Someone else on your team that has not seen the code yet gets 2 hours to fix as many edge cases as they can.

Testing design and maintainability

icefox2• 17 years ago

The more I ponder it the more I am curious just how this would turn out. Would it be like usability tests were in round 5 the team is screaming at the team member behind the sound proof glass while he fumbles around? Would each group end up with their own version of "The Little Manual of API Design" tweaked for their language/tools? Would writing tool #2 actually be harder than #1?

nopassrecover2• 17 years ago

Heh I thought this was sarcasm/satire (especially given the million edge cases) but it's not a terrible idea as you say.

cbr• 17 years ago

Or simpler: you solve problem X, making you code as clean, documented, and extensible as you can. Then a team member has to extend it to solve X', where X' is designed to show up bugs in the solution for X (turkish language insteadof english, etc).

electronslave• 17 years ago

So, basically: the sort of person who loves to fling all their mental resources at a problem is, at the end of the day, a person without mental resources. And while competition is a great way to feel like you're showing up everyone else, you're still a sweaty nerd arguing with other sweaty nerds about whose hash uses less cycles.

I see. Man, I'm glad I left academic computing.

lacker• 17 years ago

Programming competitions aren't a great reflection of either real-world programming or real-world CS research, that's true. But that doesn't mean they're a bad thing.

Here's some pro-competition points.

1. Any way of making programming more fun will get more people into programming.

2. Competitions encourage you to learn the mathematical, algorithmic side of programming, which is another useful angle to attack problems from.

3. Competitions help people learn teamwork under pressure.

4. Competitions provide an incentive for people who are already acing their classes to learn more and improve themselves.

5. Practicing for competitions forces you to become broadly acquainted with standard algorithms.

dkersten• 17 years ago

I took part in several competitions and while I didn't do as well as I would have liked, I did think they were useful in that they were definitely fun, I got to meet some interesting and likeminded people, some of whom I'm still in contact with, many years later, it was challenging and provided motivation to learn more programming techniques, improve my skills and it helped me learn new ways of solving problems. Sure, the problems weren't something you'd often encounter in the real world, but the experience and education was worth it.

Locke1689• 17 years ago

3. Competitions help people learn teamwork under pressure.

Well said. I can honestly say that I've been in very few situations more stressful than the ICPC programming competitions. I guess I'm decent but not great at competitive coding, so I always expect more of myself than I end up doing :-/ Great experience though, wouldn't miss it for the world.

dlevine• 17 years ago

Programming competitions are essentially a means for nerds to measure their dicks. Nothing wrong with that. Many competitions deal with idealized, stylized problems that aren't an accurate representation of reality.

That's ok. I don't think that it harms the rest of us, and if it makes some of the participants feel good about themselves, a contest has accomplished its goals.

scott_s• 17 years ago

I think it's valid to complain that programming competitions stress the less-important aspects of programming, and that through stressing this, perhaps perpetuate focusing on the less-important parts.

But the other side of that is I assume the people who compete in them think they're fun. And if people think it's fun, hey, why not?

fallintothis• 17 years ago

As a competitor (albeit one who's none too serious or good), it's remarkable how precisely you've reflected my attitude about programming contests.

Here's a written sample of my numerous (or just repetitive) rants from an old reddit discussion (http://www.reddit.com/r/programming/comments/6xrpc):

I participate in the ICPC, my school's local programming contests, and, in a minimal way, this year's ICFP (I had midterms to study for and was fed up with their buggy sample server). They're very fun, and that's pretty much all you should expect. The contests are a lot more social than people give them credit for. Hanging out in the computer lab, kicking over chairs as you miss the deadline for a problem, getting friends to participate for the first time, the exhaustion of failure, the thrill of finally getting that message about passing all the tests, post-game pizza party, high-fiving each other for solving 0 problems each, going over solutions, reading each others' hilariously frustrated code ("Fine! Let's brute-force this fucker!"), exchanging tales of respective challenges on each puzzle, stories about cranking out solutions to a problem while sitting with your laptop in class, intermittent bantering about using conversation as a desperate distraction mechanism, breaking out the Binky (http://video.google.com/videoplay?docid=3796146278554348828&...) for help with pointers, offering Emergency High Fives (tm) to the crowd of fellow ICPC competitors as they shuffle out of the building, golfing down the solutions for days after the end of the contest, last-minute submission races, weighing whether to use the scant prize money to get a Scrolling LED Belt Buckle (http://www.thinkgeek.com/gadgets/electronic/7c60/) set to display a well-intentioned insult to the person who normally wins the contest, ... the list goes on.

The rankings are incidental, though most people don't seem to think so. At times it felt like our group was the only one at the ICPC that wasn't taking the contest so fucking seriously. Certainly good programmers can win contests, but winning contests doesn't make one a good programmer. The problems are short, artificial, and moreover a lot of fun! I don't quite understand the hyper-competitive students who make it their life to win the contest, as little as it counts for anything. I'm just happy to kill an afternoon messing around with puzzles and a group of friends; plus, free pizza.

philh• 17 years ago

There is another model of programming competition, which I'm a fan of: "Write a program [in domain X]. You have until T. Begin."

The time limit doesn't encourage good code, but without it there would probably be no code at all. ("I don't have time to take up another project" vs. "I can spare a weekend to work on this".) And it's not entirely negative: premature optimisation is discouraged. The open-endedness emphasises high-level design over algorithmic details.

The national novel writing month (not a competition, but the same format) guys have a follow-up, national novel editing month. I want to try something similar for programming competitions. You write something quickly, and it gets judged as an alpha. Then you get more time to clean up, add polish, maybe some new features, and it gets judged as a v1.0 final. Maybe this will encourage the right mix of getting things done and doing them well.

mquander• 17 years ago

Consider this -- if the author wrote a litany of reasons why he thought that chess was a poor way to measure one's skill at board games, and why chess doesn't train you to think creatively as well as writing novels does, and why your chess rating isn't a good metric for predicting how good you are at checkers, how would that differ from the article he's got?

I don't think that programming competitions are supposed to be "worthwhile," nor that they are intended to measure how well you can go on and make big systems in code. They exist because it's a lot of fun to compete in them, if you're of a certain inclination. Isn't that good enough?

The author states without elaboration that "Programming is not a sport like tennis or basketball, where one player/team "wins" and another player/team "loses." Well, why not? It can be if you want it to.

nopassrecover2• 17 years ago

But I guess some people use programming contests as a measure of your programming skill. The implication I got was that success in programming contests doesn't necessarily correlate with development aptitude.

akamaka• 17 years ago

I love programming competitions, and they have provided me with some of the most valuable experience I've ever had.

I spent a couple of years working on programming problems with my university's ACM contest team, and I've never had a chance to work with a keener group of hackers. We spent hours and hours a week in the computer lab practicing, figuring out difficult bugs, improving each other's code, and just working on random side projects.

It's a bit intimidating to work alongside world-finalist programmers, who sometimes seem to be able to pull solutions out of thin air, but after a while, you start to pick up on how everyone on the team has a unique and valuable way to approach problems.

And that's the most valuable thing. Since I've left school, I've learned great stuff about business, design, people, and usability. But I'll never again have such a pure immersion into the art of coding.

bravura• 17 years ago

[disclaimer: I was on the USACO team and my collegiate term got 8th in the world in the ACM competition]

One of the most valuable things I learned from team programming competitions was programming discipline, by writing code on paper. Yes, I literally wrote code on paper. In the ACM competition, there is one computer and three team members. So you divide up the problems and write code on paper while someone else is in front of the computer.

Writing code on paper leads you to great discipline. You write everything, so you are biased against long ugly solutions that seem simpler in your head. It turns out that these solutions are usually harder to debug. By intentionally limiting your coding speed, you end up flexing your design muscle and learning to get things right the first time. This allows you to hold more abstractions in your head, which is an important programming skill.

Naturally, there are many bad habits you could form from programming competitions. But this is one particular good habit that I am grateful for, to this day.

plinkplonk• 17 years ago

"One of the most valuable things I learned from team programming competitions was programming discipline, by writing code on paper. Yes, I literally wrote code on paper."

" By intentionally limiting your coding speed, you end up flexing your design muscle and learning to get things right the first time. "

Interesting how this matches Dijkstra's method of writing programs. I believe he additionally proves correctness before typing it in.

CrLf• 17 years ago

I participated in some ACM programming contests when I was a student. My university had about half a dozen teams, and we had local contests and participated in national and european level competitions.

I can see where the author is coming from, but I can't agree with him fully.

Programming contests really are no measure of how good a programmer is. At most they are a measure of how good a programmer is at memorizing solutions to "standard" classes of problems and then quickly adjusting them to specific instances of them at the competitions. But I don't think that's why competitions exist at all.

I admit, I was never a good programming "athlete". I like solving those kinds of problems, and I like to solve them as quickly as I can, but I also like to immediately go back to them once solved, to "clean up" the code and make sure I can understand my solution when I read it again a few years in the future (which will probably never happen, but nevertheless). The problem is the "as quickly as I can" never was "9 problems in 5 hours", not even 5 problems in 5 hours. And the same applies to my fellow teammates, which I don't see as bad programmers at all.

In fact, looking back, it is funny how I seemed to like to pick apart other people's solutions when they happened to solve the problem as far as the automatic judge goes, but then seemed to have some corner case problem where it would probably fail (incomplete solutions). This happened from time to time, and it could also be attributed to me failing to solve the same problem, but incomplete solutions really irked me. :)

But the social aspect of the thing was important, probably more important than the actual programming, not only because of the visits to other academic institutions, but also between the teams of the same university.

Why I hate programming competitions

Related Stories