Show HN: Proofville, A Competitive Editing Game – Crowdsource Your Proofreading

by lmeyerov • 13 years ago

36 16 13 years ago

ams6110• 13 years ago

Spell and grammar check have been part of any decent word processor for quite a while now.

But they can't deeply check style, clarity, or other such qualitative aspects. Word won't tell you when your sentence is difficult to read or confusing.

lmeyerov5• 13 years ago

We've caught a lot that word processors haven't, should make a gallery :)

svetha• 13 years ago

I absolutely love this. This is about way more than spell and grammar check. It is not just proofreading. Now the crowd can be a test audience to help make content more readable and digestible. I hope this works out!

ivan_ah• 13 years ago

This is very interesting. Proofreading as a service could be used by so many content companies out there.

I guess for something long like a book it would make more sense to have the same person proofread the entire thing, but for blog posts and other text, parallelism could be very powerful.

I'm not sure how the pricing will work out though:

   $0.004/w  = $1.4/page = $350 for my book

this is 1/6 of what I paid to my copy editor. Who will work for such a low wage?

Last but not least, does it all have to be through-the-web. I find I catch a lot more mistakes on paper.

Either way, I wish you good luck with your business and I will be following the developments closely.

lsb• 13 years ago

Sometimes copyeditors fix really subtle problems, sometimes copyeditors fix really obvious problems.

This should help fix the obvious problems, and make their time better-spent!

lmeyerov5• 13 years ago

Right, looking at crowdsourcing (the main proofist site) was born out of my frustration that most copyediting comments are for simple mistakes that anyone could have caught. I want to make ours cheap enough that you can get 10-20 pairs of eyes per piece of text and without the usual wait.

This game is an experiment to make it free, and if we get mass, can get the desired level of matching. (In just one hour, we're already a hit with educators in math/science!)

gojomo• 13 years ago

Related: the Soylent Word Processor ("It's got humans inside!") from the MIT CS & AI Laboratory:

http://projects.csail.mit.edu/soylent/

lmeyerov5• 13 years ago

Definitely! Also, there are a couple of wonderful followup projects that think about how to make crowdsourcing affordable (each task still costs money): http://www.cs.berkeley.edu/~kuangc/

cdcarter• 13 years ago

Is there a way to proofread something without having to upload your own text? Now I'm in the mood to edit some copy!

lmeyerov5• 13 years ago

There is for the main site. I like the idea of different matchmaking modes for the game. A player suggested allowing more than two reviewers working together in a session, which may be one fun way to do it!

vitovito2• 13 years ago

I've been working on my own mturk-based proofreading system for OCR scans, modeled after Ryan Tate's typingpool. This is very close to a web-based version.

Your illustrations suggest you chunk content to mturk at the paragraph level, is that correct? Did you try it at other levels of granularity, by the sentence, word, or N number of words? If so, what were the results? If not, why not?

How did you settle on 4¢/word?

Are you considering a mobile app/FB version of Proofville?

What's the back-end? Is it your own system, or is it an existing crowdsourcing infrastructure like PyBossa?

How many reviewers compare a passage? 1? 2? Best of N until there's consistency?

lmeyerov5• 13 years ago

A housemate (linked below) runs http://captricity.com/ -- crowdsourcing OCR is a really good idea.

The collaborator for the main site used to work @ cap, and we've been evolving a REST API to simplify calling into mturk.

For quality, we don't use arbitrary mturkers. Getting good results on mturk takes a lot of experimentation: I now think about it is as a combination of HR agency, traffic conductor, and street vendor. The challenges of building a useful & sustainable app on it are a bit different from what's addressed by crowdsourcing research papers, though that's not so surprising in retrospect.

Sounds like this deserves a blogpost. It's been a fun summer project. Basically, nodejs with roll-our-own crowdsourcing algorithms, APIs, and UI widgets. A lot more to do, but it's finally good enough to address my proofreading needs, and I'd like to help others who also need to get some writing done asap :)

Btw, we're $0.004 / word, not $0.04 ;-) That means we already cost 60% less than shops with comparable quality and slower turnarounds.

vitovito2• 13 years ago

Yes, please do! I can only begin imagine some of the issues, having fed a couple thousand minutes through typingpool, but there isn't a lot of writing about using it at scale. I'll look forward to it.

relate• 13 years ago

Tried it out, works well!

jnbiche• 13 years ago

If this takes off, an API would be great, allowing people to integrate this into WordPress plugins, etc.

Show HN: Proofville, A Competitive Editing Game – Crowdsource Your Proofreading

Related Stories