How Crowdsourcing Turned on Me

http://nautil.us/issue/18/genius/how-crowdsourcing-turned-on-me

by dnetesn • 12 years ago

78 24 12 years ago

incision• 12 years ago

Fascinating stuff, not at all what I was expecting. Two things come to mind reading this.

1.) The crowdsourced and algorithmic solutions seem to be framed in contrast if not exclusive to each other here. I wonder if combining the two couldn't have helped avoid this:

'However, the crowd was hopeless against a determined attacker. Before the first attack, our progress on the fourth puzzle had combined 39,299 moves by 342 users over more than 38 hours. Destroying all this progress required just 416 moves by one attacker in about an hour.'

I'm wondering if it's easier to create a 'bouncer' algorithm that detects and blocks/reverts aberrant behavior than to solve the core puzzle itself. If that problem is in fact easier - identifying 'hurtful' or unusual moves, could addressing it help the core solution by helping avoid non-malicious 'back sliding' too?

(I have no clue about these things.)

2.) The sort self-importance expressed by the attacker is just awful.

Unlike the author, I don't see it as a concern about "how crowdsourcing puts the collective potential of humans above technology". That almost sounds noble.

This impresses me as plain old envy and entitlement, an overgrown cousin to every "Why is this on HN?" or "Why is this on the front page? I submitted this before and no one voted for it." comment on HN. The whole thing is one big "should" of personal gratification.

xerophyte129323• 12 years ago

The sort self-importance expressed by the attacker is just awful

While I agree that instead of behaving maliciously, the other competitor should have lauded their simple yet effective solution, I can imagine the WTF he must have experienced upon discovering their site. I mean think about it, if your team has been spending god knows how long wracking your brains to come up with CV algorithms to solve this stuff, and then you realize the team in the lead isn't even using an algorithm! They are just using a huge amount of people to solve it! Now of course the right reaction to this should be to laugh wholeheartedly and ponder about how a motivated crowd can trump advanced technology.

We had a similar WTF moment during Google Code jam 2011. In the qualification rounds, there was a problem Goro Sort[1] and the way the question was written, it appeared to be a very tough problem. Later we discovered that one of our friends solved it with a VERY simple algorithm: just count the number of elements out of place! and it WORKS!! The good part was, instead of being jealous, we did laugh about it and still do

Thimothy• 12 years ago

In my opinion, the attacker missed completely the point of the contest. Its aim wasn't to know who was the one who could come up with the most clever visual processing algorithm, but knowing which is the best way to solve that particular problem, and "Through computer-aid crowdsourcing" could very well be the best answer.

I really hope he wasn't in the winning team.

probablyfiction• 12 years ago

The ability to ban rogue users would likely have helped them immensely. An invitation-only system would probably have been of benefit as well.

Nothing is foolproof, obviously, but their users were working quickly. They only would have needed to keep malicious users away for a limited time in order to achieve success.

The author likely doesn't have much real-world experience with users or his team would have built this sort of capability into the software from the ground up. Any dev worth his or her salt knows to never trust any user.

sk8ingdom• 12 years ago

And one of the crowd-sourcing examples, Wikipedia, has this built in functionality. Certain Wikipedia pages are under constant attack and diligent contributors do what they can to keep them accurate.

patrickphilips• 12 years ago

'In retrospect, it might have been foolish to assume that every member of an anonymous crowd would act according to our best interests.'

In my experience with crowdsourcing, most people actually will complete your task honestly and genuinely. Some may not understand it, but they generally just give back noise and cancel each other out. You'll almost always get a few malicious people though, even if their motivations aren't as straightforward as this case.

If you forget to account for this adversarial subset of folks, eventually you're gonna get got. Very cool story/use-case though.

GhotiFish2• 12 years ago

This was a good read, but the author is not above narcissism it seems.

   He reminded me of the Luddites in the 19th century, who 
   destroyed the cotton- and wool-processing technology 
   that they feared would replace man with machine. Only in 
   this case, the concern was reversed: The attacker seemed 
   to dislike how crowdsourcing puts the collective 
   potential of humans above technology.

or sometimes extreme narcissism.

   “artistic flare.” The words are now what I imagine when 
   trying to express the struggle between the creative, 
   collective mind that emerges online, and the dark, 
   almost eerie forces that antagonize all kinds of genius.

In my experience, mobs like 4chan tend to descend on people who have high and mighty ideas of their actions. Though almost certainly in this case it was just because the tool was vulnerable to such activities.

The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?

Still, like I said, good read.

dsuth• 12 years ago

In my experience, mobs like 4chan tend to descend on people who have high and mighty ideas of their actions.

In my experience, people who defend 4chan's actions like to victim-blame.

The authors inability to empathise with "... and personally feel that crowdsourcing is basically cheating (and I’m not the only one that feels this way)." is something that makes me leery as well. That's an understandable viewpoint, how is this going to work with actually classified documents?

Actually this line (from the attacker) garnered the most attention from me. So the attacker felt that crowd-sourcing the solution was cheating - but maliciously attacking and sabotaging an opponent in a competition was just fine? This sense of entitlement is very typical of anti-social b-tard behaviour, and it shouldn't be encouraged.

Personally I find the tension between clever algorithms and crowd-sourcing very interesting. Applications like Foldit for protein folding produce some amazing results. Sabotaging valid solutions does nobody any favours, except in an extremely selfish and short-term way.

GhotiFish2• 12 years ago

but maliciously attacking and sabotaging an opponent in a competition was just fine?

Empathise does not mean condone.

This sense of entitlement

entitlement? Is someone acting entitled?

very typical of anti-social b-tard behaviour,

nice...

and it shouldn't be encouraged.

and it can't be stopped. It is a reality.

xerophyte129323• 12 years ago

I don't think Darpa was looking for an algorithm to unshred documents, as the article states, they were assessing vulnerability of shredded documents. In that regard, this team's efforts were most important as they highlighted how an adversary doesn't even need a complex CV algorithm, all they need is a huge crowd and an incentive

xxs3• 12 years ago

That's an understandable viewpoint, how is this going to work with actually classified documents?

My thoughts exactly - the system is designed for the military, now imagine getting thousands of random people peeking at the documents.

angersock• 12 years ago

In other words, creation took 100 times as many moves and about 40 times longer than destruction.

So, yeah, pick your new engineering hires veeeery carefully.

JackFr• 12 years ago

So this is like a speeded up Wikipedia.

semiel• 12 years ago

Really interesting article. I think we've only begun to scratch the surface of what's possible with crowdsourcing.

To me, the author's failure seems to be a straightforward one of design. The system allowed attackers to ruin everyone's work, and to do so in a much shorter time than the positive work took. It seems to me that it should be possible to design a system that doesn't have these properties. (Of course it's probably not trivial to do that without making it harder for honest people to participate. But that's a design challenge, not a fundamental flaw of crowdsourcing.)

Jemaclus• 12 years ago

Just as an exercise in 20/20 hindsight, it seems to me that the easiest change to make is to not trust every player implicitly. In other words, you enforce some level of duplication in the work. If enough people agree that this piece goes there, then it goes there, and if not, then it doesn't. A single attacker is unlikely to be able to undo the work of dozens of supporters, even if it takes less time and effort to undermine the truth.

I believe Foursquare did something like this with regards to their recommendations. They'd ask questions about locations (e.g., pick a beautiful picture, is this place classy?, is it loud?, etc), and only when enough people agreed would they use the information in recommendations.

hobs• 12 years ago

Yeah that is what I immediately thought, that you would want to do some processing on the changes before presenting it to the users, and not make it live, just asking them to make a change and asking others to rate the change.

That would be a system that you would need a large number of users to manipulate.

The downside of this idea is that their website is clearly about an interactive experience where you move things around and get the feel of others doing the same thing, I dont know if that kind of thing would be compatible with y/our idea without a lot of tricks.

Animats2• 12 years ago

Bad design. Anybody could mess up the job. In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or a MMORPG, or a factory.

Given the problem they were solving (reassembling images shredded into strips) they needed something where, when two strips were put together by a human, a program decided if they matched closely, and, if so, locked them together for further editing. That would constrain the workers to make forward progress. You might need an unlock capability if there was a false match, but that should take the cooperation of several people.

skybrian• 12 years ago

Or just versioning and the ability to revert, like Wikipedia has.

xxs3• 12 years ago

They would need huge amounts of versions and then need someone manually to observe. With a single attacker would be easy, though.

clockwerx• 12 years ago

Probably better is something like tasks.hotosm.org's "Done" and "Reviewed" approach, and requiring accounts.

Obvious vandalism attracts a ban quickly.

xerophyte129323• 12 years ago

In any large, anonymous group, there will be a certain number of assholes. Of course it didn't work. Ask anyone involved in running an IRC channel, or an MMORPG, or a factory.

My favourite example: Twitch plays Pokemon

xxs3• 12 years ago

if they could do the matching they would have not needed the human force.

Animats2• 12 years ago

Doing all the matching automatically is reasonably hard, but merely checking a human match isn't.

KaoruAoiShiho• 12 years ago

Anyone remember reading something very similar to this about a year ago? It might have been the same situation but I distinctly remember an article about crowdsourcing failing due to an attack.

How Crowdsourcing Turned on Me

Related Stories