Users of my iOS Game Teach Me a Lesson MIT Didn’t

http://blog.aaroniba.net/2011/07/06/a-lesson-from-my-ios-users-they-dont-teach-at-mit/

by aaroniba • 15 years ago

186 77 15 years ago

blog.aaroniba.net

geuis2• 15 years ago

tldr: MIT doesn't teach everything. Author discovers rapid iteration, the value of throwing away code, and possibly source version control because users of his game solved it faster than he did with superior "logical" reasoning.

apike• 15 years ago

Don't take this so lightly. I've seen a lot of brilliant engineers, especially new CS grads, get stuck in this hole. They have the intelligence and knowledge to "correctly" solve problems, and are driven to always to get them right the first time.

These grads can have a hard time "just fucking doing it", especially when it comes to getting started on very hard problems. It can take a few years of real world experience for them to get comfortable with doing something that's known to be wrong and moving from there.

geuis2• 15 years ago

Excellent point. The comment wasn't a slight against the author. Far from it, I wish I had a mind even 25% as analytical as his.

palish• 15 years ago

I don't understand. They know how to do things, yet they can't do things?

amirmc• 15 years ago

I appreciate the tl;dr though since I've got the gist of the post and decided it's not one I want to read right now.

bh422222• 15 years ago

I'm not surprised. Even my no-name state university did not teach source control or any editor/IDE skills. I picked all of that up from the other students and learned even more by coding myself into a corner, fun!

But that's all because I knew I wanted to be a practicing programmer, not just a computer scientists.

A lot of university cs programs have this attitude that practical knowledge (and I'd argue that source control is beyond practical, it's essential) is yucky and should be left to the technical and vocational schools.

But things like iterating and heavy editing and re-writing are in no way programming specific, the exact same principles apply to writing plain old English. Does anyone know if English departments teach these things?

ohyes• 15 years ago

Iterating, heavy editing, and rewriting should be taught before you get to college.

Really they are the most important part of any sort of English composition. Getting words on paper is easy, getting the right words on paper is hard. As soon as you begin writing papers for school (in middle school, perhaps), you should be learning all of those skills.

If you don't know them by college (which is entirely possible), they might be offered in a remedial 'English composition' course.

trentfowler• 15 years ago

The author is one cocky guy. This article mostly annoyed me.

>I figured that I, the author of the game, with my degree in mathematics from MIT, and years of puzzle-solving experience, would be much better at solving my own puzzles than random users downloading my app would be.

>I thought there was a mistake because some users were solving puzzles faster than I was! Were they cheating?

pc• 15 years ago

Anyone who knows Aaron will tell you that he's actually pretty humble. I think it's easy to imagine the thought process without it being especially cocky: "I built this game and have spent more time solving these puzzles than any of these guys. How are they so much faster?"

_debug_• 15 years ago

OK, Aaron.

vecter2• 15 years ago

I think it's safe to say he was being tongue-in-cheek.

pluies_public• 15 years ago

I actually got the exact opposite feeling: he was cocky at first, but then realized he was wrong. And even wrote a blog post about it. That's pretty humble. :)

16s• 15 years ago

His claim seems reasonable. I'd bet on him ;)

I liked the article a lot. I'm an "iterate and repair" guy but am trying to use careful thought, logic and planning up front more often. I think that the value of the article is that it shows that neither approach is perfect by itself.

mcphilip2• 15 years ago

Good thing he didn't make a sudoku app or he wouldn't have learned the same lesson... good luck taking an iterative approach to that.

Seriously though, sometimes it's good to approach a problem using that advice from the chess teacher in Searching for Bobby Fisher: "Don't move until you see it"

http://www.youtube.com/watch?v=V36gt6_uJy4&t=7m34s

ebiester• 15 years ago

Oh, iterative approaches often are used when stuck in sudoku. deduce everything you can, but once you can't, guess where there are the least options remaining. erase if you can't get there from here.

mcphilip2• 15 years ago

Sure, you can guess at Sudoku if you want (though it's often considered poor form to design a puzzle that requires a guess), but the (intended) point was that you don't jump right in and start iterating from the beginning.

_pius• 15 years ago

Mentioning MIT makes this a nice, low-brow analog of "Worse is Better" by Richard Gabriel:

http://en.wikipedia.org/wiki/Worse_is_better

sc68cal2• 15 years ago

Yes - this is exactly the advice that BWK gives. Rule #2 specifically.

    Keep it simple.
    Build it in stages.
    Let someone else do the hard part.

Sourced from: http://rixstep.com/2/2/20080125,00.shtml

pyre• 15 years ago

  > Let someone else do the hard part.

Obviously not everyone can follow this strategy or none of the hard parts would get done.

gjm11• 15 years ago

Unless different parts are hard for different people.

sc68cal2• 15 years ago

You're missing the meaning. You should be using libraries and other small tools, to assemble more complex and powerful tools. Not reinventing the wheel. That is the way of Unix.

tmgrhm• 15 years ago

As a designer (and soon-to-be programmer) I totally agree that iteration is key. Iteration is so important to producing a polished product with the best possible aesthetics and experience.

However, I really do not agree that the right approach is to not think about the best possible design beforehand. Ramming your fingers in there and fumbling around blindly is a terrible idea. Think beforehand about the best possible solution and then repair and iterate on that.

Apocryphon• 15 years ago

I suppose MIT students don't learn guess and check.

zeteo7• 15 years ago

What he calls "Iterate and Repair" is basically a greedy algorithm. Greedy algorithms are great where they work, but there are also situations where they break spectacularly. I guess the algorithms course is no longer required at MIT?!

richcollins• 15 years ago

How is it a greedy algorithm? He didn't say anything about the "intuitive" decisions maximizing immediate gain at every decision point.

zeteo7• 15 years ago

I'm sure the users he describes can be trusted to make locally optimal choices.

aaroniba2• 15 years ago

Most greedy algorithms I've seen do not have a repair step. But whether I call it "iterate-and-repair" or "greedy" doesn't change the message, does it? Or are you saying that I shouldn't be surprised that iterate-and-repair works so well on this puzzle?

zeteo7• 15 years ago

What I'm saying is that you shouldn't draw such general conclusions from seeing a simple algorithm work in a particular case. Your problem-game turned out to be algorithmically simple. This probably makes it more appealing as a game. But most things in life will not be as simple, and this includes large coding projects.

What you call the repair step I see as running the greedy algorithm again, from a different node in the search graph.

knowtheory2• 15 years ago

Really?

OP isn't right that this is greedy either. What your users are doing is an iterative search with a backoff strategy. They pursue one path through the search space, and when they don't find the solution, they remove some links (backing off) and try a new path through the search space.

The clever bit is whether you can automate their decision mechanism for how they're proceeding through the space step by step, and what points you can fall back to when something does wrong (and how to recognize when you've gone wrong).

The fact that people can do this by the seat of their pants somewhat shouldn't come as a surprise to people who study cognitive psychology, but certainly might be a surprise to engineers. After all, that's why everyone who's not a social scientist loves Malcolm Gladwell's books.

gosub• 15 years ago

"iterate and repair" is a pretty standard form of local optimum search, like in http://en.wikipedia.org/wiki/Hill_climbing or simulated annealing. Why are you surprised that works for exploring the decision tree of a puzzle game?

alanh2• 15 years ago

Regular expression search is often greedy and also repairs by next examining a less greedy partial solution

sitkack• 15 years ago

"Quid in nomine? Quod dicitur rosa Odorem suavitatis, ut non alio nomine.

All you guys are trying to label, taxonimicate the "lay person's" strategy into your hierarchy.

Humans are damn smart, and the cocksure MIT nerd got his ass handed to him.

dev1n• 15 years ago

this is true for the most part. Humans have always been the "try it and if it kills somebody try it another way." i.e. we used to eat tobacco leaves. Now we smoke them. riddle me that.

caf3• 15 years ago

It sounds a bit like simulated annealing.

shaggyfrog6• 15 years ago

Simulated annealing is a probabilistic algorithm.

The state space for this game is not stochastic. It's deterministic.

Standard single-agent search techniques apply here (i.e. A* and its variants) and essentially encapsulate this "iterate and repair" idea (especially IDA*).

caf3• 15 years ago

It doesn't sound like the users mentioned are using a completely deterministic algorithm to determine the next solution to try, though - it sounds more like they're making random tweaks. Albeit the tweaks are guided by intuition, which makes it somewhere in between the two - neither completely deterministic nor completely unguided.

dpcan• 15 years ago

Every programmer could benefit from taking creative writing courses in college.

You are forced to purge your mind of every disorganized thought onto paper, and then go back through as often as necessary until it's right.

This may not be a great "programming" strategy, but it certainly makes the design process more fun and productive.

blahedo• 15 years ago

One of my students (CS) once took a creative writing class, and her professor, a man well-known in the creative writing community, told her he thought computer programmers make better writers. She was pretty sure this had a lot to do with understanding revision and outside input. When you write a program, the compiler (or possibly your unit tests) tells you "it's wrong! Fix it!" and doesn't always give a lot of guidance as to how, but you figure it out and come out the other side with a better product. A lot of the other students in the class took the critical comments of the professor (and peers) as almost personal attacks on their perfect creation.

All of which is to say that the Computer Science/Creative Writing benefits may go both ways.

MaysonL• 15 years ago

Writing is 90% editing: so is programming.

da5e• 15 years ago

When I was a kid I had the top score in Galaga at my local arcade. I held the spot for months. Then one day I came in to find my score doubled by someone. The arcade owner told me that it was a guy who went around to each arcade and ran up the scores. Like a game playing gunslinger.

Now on the web any time I find a fun game and play it for awhile and improve, I come online to find that the high score has been run up by some aspieoverlord to unwinnable levels.

So how do game designers overcome that demotivating unwinnable high score phenomena?

zoudini• 15 years ago

Personally I'd love to see replays of "impossible" scores. Similar to the speed run videos for games like super Mario on YouTube

monkeypizza2• 15 years ago

There should be 3 levels of high score lists should be 1. personal 2. friends 3. global

this actually applies everywhere you are showing group merged data related to a particular person.

i.e. your height, your friends heights, and global heights

imagine a news article that has interactive graphs that query g+ or fb and use the info to put a friends layer next to the global info.

chopsueyar• 15 years ago

Instead of social circles, how about geographical locations?

1. local (city or neighborhood)

2. statewide

3. country

4. earth

It is more analogous to the parent comment.

coryl• 15 years ago

As someone who has used trial and error processes my entire life to figure out problems, am I missing out by not being a deliberate planner?

danking00• 15 years ago

Forgive me for I am a student of HtDP and not SICP, but when this author was attending MIT wasn't 6.001 still in place? The Scheme REPL, which I imagine was introduced to students of 6.001, encourages an iterative approach to problem solving.

Perhaps this author didn't learn about iteration from MIT, but implying that MIT is entirely ignorant of this seems unfair.

edit: formatting

Legion• 15 years ago

Or possibly he wasn't being so pedantically literal.

ericflo• 15 years ago

This is exactly how I solve everything, and it's why I'm so bad at programming on a whiteboard in interviews.

sev• 15 years ago

The used method reminds me of randomized sorting algorithms and how they're often more efficient than deterministic methods.

tylerneylon• 15 years ago

This reminds me of the famous Knuth quote: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil."

In this case, I'd say the premature optimization was simply trying to prove the correctness of each move before it was made. In the bigger picture, programmers have a meta-algorithm for how they solve problems. It's tempting to sit down and figure out an O(1) or maybe O(n) algorithm - whatever is the fastest possible - and only then start to program. In reality, many of these optimizations, especially on a life-scale, can be counter-productive, because the cost of designing, implementing, and maintaining them far outweighs the benefits. One of the hardest parts of good engineering is true simplicity.

So I'd say Knuth's suggested philosophy applies to problem-solving at large, and this is one perspective on the author's epiphany.

shaggyfrog6• 15 years ago

Eliminating invalid moves is not a "premature optimization", but is at the heart of heuristic search. By eliminating impossible/invalid states, you eliminate them from the search space, which not only speeds things up, but prevents the discovery of an invalid "solution". Put another way, discarding invalid moves reduces the branching factor of the problem -- a fundamental optimization.

monkeypizza2• 15 years ago

This game is similar to Mortal Coil (flash frontend to a programmable backend)

http://hacker.org/coil/

But that allows automated solutions. As far as I know, all of the best solvers there use a greedy algo, although there are some cases where you can prune massively by using logical deductions as well.

alanh2• 15 years ago

Reminds me of this strategy for finding Hamiltonian circuits in graph theory. http://alanhogan.com/asu/hamiltonian-circuit/

Users of my iOS Game Teach Me a Lesson MIT Didn’t

Related Stories