MTurk + Google News API = press

http://customerdevlabs.com/2013/09/24/google-news-api-mturk-press/#

by langoff • 13 years ago

267 111 13 years ago

customerdevlabs.com

caublestone• 13 years ago

Sounds like you have found a potential product.

austenallred5• 13 years ago

Seriously, I'd pay to use that if it were pretty cheap.

PanMan• 13 years ago

The link to the resulting press (in the article) is broken. It's http://bitly.com/bundles/justinwi/1

justinwi11• 13 years ago

Thanks for the heads up! Fixed.

seeingfurther6• 13 years ago

Are email addresses for the reporters on every article? I'd be interested to know what the stats are.

justinwi11• 13 years ago

There are rarely email addresses on a particular article, it's usually a matter of tracking down the author and finding his/her email address.

We ended up getting addresses for about half of the article authors.

seeingfurther6• 13 years ago

So the the Mturk worker would check the article for the email, if it's listed great, if not they might Google it?

yahelc• 13 years ago

You might find this service I built useful: It queries the Bing API for different combos of the name/domain (John.Smith@nytimes.com, JSmith@nytimes.com, etc) and returns their counts, and then deeplinks into a Google verbatim search of that email address.

http://emailguesser.com

Source is on Github: https://github.com/yahelc/emailguesser.com

randall• 13 years ago

Great hack! Very impressed.

eric_cc• 13 years ago

Came here to say the same. Thanks for sharing.

alopes• 13 years ago

Does MTurk work outside the US?

mahranch2• 13 years ago

It used too, but they've had a lot of problems. Apparently there were a lot of scammers from India so they (temporarily?) ceased allowing new accounts from outside the US.

knockonthedoor• 13 years ago

"The Google News Search API has been officially deprecated as of May 26, 2011."

justinwi11• 13 years ago

True, but deprecated != dead.

API still works great. You can try it yourself: http://press.CustomerDevLabs.com

Humbled• 13 years ago

Did any of those 300 emails bounce. I.E., how many weren't correct?

Edit: Not yet tried mTurk so just curious how legitimacy and payments are handled.

justinwi11• 13 years ago

Good question. Overall quality was high (you're able to see the results as they come in and if they look funky, you can reject them and you won't have to pay).

Stats on our campaign are here: http://screencast.com/t/5XizDvsvjP

mmaia• 13 years ago

Really nice!

That's pretty much the process I used to launch some apps -- except that I like to get the contacts myself and read some other articles from the person, so I can add a personal touch to the emails (it will increase your chances with the top journalists).

After successfully using this method, I'm building http://get.press.io

jamesjyu• 13 years ago

Very awesome hack.

beNjiox• 13 years ago

Great! You are my Hero

tomgruner8• 13 years ago

Mechanical turk is abusive. It bypasses the laws that were designed to protect the most vulnerable, and it will take some time for new laws to catch up that can regulate operations like mechanical turk. Just because new services and start ups have out maneuvered the law making system does not make those services ethical.

"Workers relying on such low wages and unstable employment are not likely to be able to educate their children enough to escape increasingly high rates of unemployment. A sustainable form of crowdsourcing will require forms of collective governance that mitigate the effects of market competition on those treated as mere links in a chain of algorithmic logic."

http://economix.blogs.nytimes.com/2013/03/18/the-unregulated...

Workers will do jobs for as little as 25 cents an hour. http://www.thedailybeast.com/articles/2011/06/22/americans-w...

According to the nytimes article, 70% of the workers are women. For me, the possibility of paying an american woman 25 cents an hour so that I can get some press coverage is disgusting. When we needed data entry done, we hired some of our out of work friends for about $12-$15 per hour. It served our needs just fine and cost us about $1500 total.

For fast turn arounds, I see why you would want mechanical turk, but planning ahead can give you the possibility of treating people ethically. I don't agree with any systems that perpetuate poverty. Mechanical Turk obviously provides a very valuable service to businesses however. Does anyone know if there any systems like mechanical turk that guarantee at least minimum wage is being paid to the workers?

robrenaud• 13 years ago

It's not as though Amazon is going and pointing a gun at people's heads and telling them to Turk or die.

If it's a bad trade for people to exchange their time for the low wage, they shouldn't take it.

sp332• 13 years ago

This argument sounds good on paper, but it has centuries of evidence against it. http://en.wikipedia.org/wiki/Trade_union#History

tomgruner8• 13 years ago

Rob, as a software engineer at google, I would not expect you to to be able to relate to those without opportunities. In fact, there is some evidence that utilizing your brain to solve technical problems can restrict its ability to empathize. Of course amazon has no gun, but that does not mean that poverty in itself is not violent. As you like science and data, here is a read for you: http://www.sciencedaily.com/releases/2012/10/121030161416.ht...

gnaritas8• 13 years ago

If it's a bad trade for people to exchange their time for the low wage, they shouldn't take it.

They may have little choice. You don't seem to understand what it means to be broke and unskilled.

justinwi11• 13 years ago

Love your passion social justice. I share it, and I can understand how a cursory understanding of mTurk may feel exploitative.

That said, it's clear neither the NYT nor the Daily Beast understand MTurk users - and because of that, they mistake a hobby for a job.

For many people in the US, MTurk isn't about making money, it's about doing something while you're bored. It's not a replacement for a job, it's a replacement for Farmville.

"Ever wonder what our labor market would look like without minimum wages or labor law protections?"

If Turkers were paid with a virtual currency, there would be no comparison to minimum wages - but because they're paid with cash, the puzzles/work exists in a weird world between work and play.

Also worth mentioning, I intentionally price my HITs low so as to target turkers who aren't looking to supplement their income. It only takes 5 minutes of trying to find one person's email address to realize $.03 isn't worth it - if your goal is to make money.

I may be still missing the boat, but if Turkers really are doing this as a hobby - is it still exploitative?

tomgruner8• 13 years ago

Hi Justin, that is a good point. According to this study in 2010 on page 4 http://sites.uci.edu/andrewzaldivar/files/2012/05/SocialCode... about 82% of the survey responders answered that the income was not essential, but the other 18% responded that the income was either important or essential to make ends meet.

Doing some turking to buy extra things while someone else pays your bills isn't exploitive in my opinion. But, for the 18% who are using is to make ends meet I do think it has a very real possibility to be exploitive. That's where regulation could help, in ensuring for those people who really need it have the safeguards in place to prevent the exploitation. I am far from an expert on Mechanical turk, and if it could guarantee that minimum wage was being met then I would not be against it.

jmduke• 13 years ago

(Disclaimer: I work for Amazon, but not on anything related to mTurk)

Is there reason to believe that people are using mTurk full-time as a significant source of income? I've Turked a bunch -- and know people who do it for a couple hours a day -- and my understanding is that its overwhelmingly used as a way to pick up a few extra bucks while catching up on TV, a slightly more civilized replacement for clipping coupons or filling out paid surveys.

tomgruner8• 13 years ago

According to the same 2010 study I mentioned in another comment, about 7% use it 30 hours per week plus. That's not a huge number but I imagine that those of the 7% would consider it full time work. http://sites.uci.edu/andrewzaldivar/files/2012/05/SocialCode...

pessimizer2• 13 years ago

http://www.mitpressjournals.org/doi/abs/10.1162/COLI_a_00057

"Amazon Mechanical Turk: Gold Mine or Coal Mine?" from Computational Linguistics.

entropyneur• 13 years ago

Care to elaborate what is the exact mechanism by which MT perpetuates poverty? Given two worlds, one with MT and one without, what exactly are the opportunities that poor people are deprived of in the former which are available in the latter?

NolF• 13 years ago

Basically, by spending so much time doing work for peanuts to make ends meet or as a primary source of income, they become reliant on that income and will not find opportunities elsewhere. The argument is that mTurk perpetuates and thrives on cheap labour.

The solution is obvious, make the decision for them and deprive them of that source of income or legislate minimum wage rates... This in my opinion would be very shortsighted and harmful.

vincvinc• 13 years ago

Does anyone know if there any systems like mechanical turk that guarantee at least minimum wage is being paid to the workers?

I think that was the whole point behind Mobileworks, don't know how well they are doing though. https://www.mobileworks.com/company/fair-trade-work

anandkulkarni• 13 years ago

We're doing quite well, as it so happens.

Paying fair wages means better work output, which means you can get higher quality results, and actually means lower total cost. It's better business than trying to scrape usable results out of bottom-dollar markets like Turk, and far easier, to boot.

We're growing fast and hiring: https://www.mobileworks.com/careers/

vdaniuk• 13 years ago

Your attempt to invoke strong emotions may be a useful rhetoric instrument, but it is absolutely not adding anything to those doing the jobs at MT.

How in the world can you even waste your time on criticising use of a high-tech product that is available only to those with a computer and access to internet? Surely there are those who are exploited much much more than users of MT? Especially, if your own link shows that 82% of survey responders answered that the income was not essential.

This is game theory all over again. If "ethical" people wont use MT, only the bad guys will reap the benefits of the increased efficiency. Shaming people doesn't help much if the "right" option is detrimental to their incentives.

Moreover, your US-centrism is showing in full force. Don't you think that paying an Indian or Romanian 25 cents/hour is disgusting? Ok, you hired some of your friends for $12-15/h for mechanical job. So ethical. Why didn't you do that in an English-speaking part of the world where citizens live for less then $1 per day. Wouldnt it be more ethical?

If you are concerned about education of poor children: spread knowledge about moocs, help with youth coding camps, tutor people or do free seminars.

And if you are really-really concerned about the Mechanical Turk then by all means go ahead and teach people who are on MT how to earn more, that would be so much better than complaining on HN about the ethics of MT.

tomgruner8• 13 years ago

I am not trying to shame anyone, and yes, I am involved in researching solutions for a project that aims to help people in poverty empower themselves through providing data processing services. I also accept lower wages to be able to work on projects that are beneficial socially and have rejected offers to work in industries like high frequency trading that were very lucrative. I have spent hundreds of hours volunteering in my life teaching reading, ged, and esl in low income and even volunteered to teach ged material in a jail. We can all help in many ways, but the main point is having good regulation and rules is beneficial to capitalism. The best solution would be proper regulation of mturk to make sure that all businesses are on a level playing ground, whether they use mturk or another solution that at the moment may be more expensive in order to ensure minimum wage. But until that regulation happens, or mturk ensures minimum wage, I won't use mturk for any project.

blablabla123• 13 years ago

25 cents per hour is roughly 45 $ a month. (8 h/day, 22 working days/month) I guess someone who would work for that wage lives in a rough country. As we all know from the news and documentaries, those countries often have horrible working conditions, like 60 hours or more a week and physically hard work. Isn't a mTurk job better than that? Also don't forget the lower prices in such regions.

The thing is this: NGOs and friends are trying since decades to solve humanitarian problems. Unfortunately they suck incredibly at what they are doing. The countries stay poor, the working conditions stay bad, war and crime continue.

I strongly believe that economical prosperity automatically leads to better living and working conditions. Just look at the African countries the Chinese got their hands in.

BTW, don't forget that there is not only competition between the workers but also between those submitting jobs. I.e. many mTurk users => more competition on the income side. Even better: competition with same working standards. You don't even have that in the U.S.!

Dylan168073• 13 years ago

The comment you're replying to very clearly focused on such a wage as being bad in the US. Sure, if the worker is somewhere that 25 cents an hour is livable, then everything works out okay.

patrickphilips• 13 years ago

$0.03 per email seems awfully low, and if you do 1 article per minute, you're making $2 an hour, which is definitely not cool.

But how long does it actually take to find one email from an article? Maybe 10-15 seconds? Then you're talking about $7-$10 per hour, which for unskilled work doesn't feel especially exploitative.

The real draw of systems like mechanical turk is not that labor is cheaper (it usually isn't), but that's it's more flexible, i.e. you don't have to hire and train people for any particular task.

bcoates2• 13 years ago

Are the small number of people depending on Mechanical Turk for real money actually capable of doing regular minimum wage labor? We already have broad exceptions to minimum wage laws that allow the marginally employable (seriously disabled, mostly) to work for less than customary sums of money while also collecting disability benefits.

Having someone who's medically unable to work or who is primarily a full-time caretaker have a home job that makes them a supplemental income on the side, even at $.25/hr, doesn't sound like an unethical or abusive system to me.

(also, turkers who are doing transcription a lot probably know how to do 4x pitch corrected fast forward on their audio player and have a good enough ear for voice to pick out the odd words)

nerfhammer2• 13 years ago

The substantiation of the $0.25/hour claim is that they were able to find three people who were willing to work for that much on one task, once. I think this is extremely weak evidence to base a lot of outrage on or to make a claim of what the market hourly rate is on MT.

espadagroup• 13 years ago

As one data point, I spent a while during college and took Turking seriously as a source of income and averaged about $7-$8 an hour over that time period. Not awesome, but not bad either for an always available, willing, form of employment.

aantix2• 13 years ago

"Always available" and and incredibly flexible form of employment. There aren't many jobs where you can sit down, make some money, when you need to take a break (kids screaming, etc), you can stop, resume again when you're ready. The flexibility is worth something.

aantix2• 13 years ago

Certainly they performed jobs for 25 cents an hour, but that rate just isn't sustainable if you actually want to get fast, reliant results.

Just as the requesters on MTurk are smart, so are the workers (aka Turkers). If you hang around the market place long enough you will quickly discover that your time is worth much more than 25 cents an hour.

The Turkers are smart; high throughput is the only way they are going to make any money.

As a requester, someone posting work, if you don't respect that in terms of 1) streaming-lining your instructions to simplify the work for the Turkers, and 2) giving a descent wage relative to the work needed

The Turkers will go elsewhere and your results are just going to trickle in.

MTurk is actually a perfect marketplace, as a worker you can see ALL of the jobs that you are qualified, all of the competing bids for a piece of your time in one consolidated place, and you can choose how and when to work instantly or whether you want to work at all.

umsm• 13 years ago

Why not pay the workers on mturk a reasonable rate of $12 - $15 an hour?

Isn't the wage set by YOU, the employer?

Stratoscope• 13 years ago

Here's where I get confused by this economic justice:

I have a little personal project and there's a piece of it that could be hired out on mechanical turk.

The project isn't worth much much money to me, it really is just a point of personal interest. And the little task is interesting enough that a few people might actually do it for free.

So I could ask for volunteers. Worse things have happened: how many times do people fill out surveys completely for free?

Surely no one would say I'm exploiting anyone by allowing them to volunteer their time? After all, it's their free choice if they want to waste a few minutes on my silly project.

But apparently I would be exploiting them if I offered a modest sum as a small "thank you".

Why?

victorf• 13 years ago

Cognitive bias is human nature. The example I read is you'd get many more lawyers working pro bono for charity than you would offering a low wage. This of course extends to our perception of other people doing it.

I for one think we should remove minimum wage entirely but give everyone adequate food and healthcare. If people didn't go bankrupt after a serious injury and had enough food to wait out crappy offers, an appropriate wage would appear naturally.

Dylan168073• 13 years ago

It wouldn't be exploitative if the money was merely a thank you.

But the money wouldn't be merely a thank you. Not for everyone.

If you were making a standard offering on mechanical turk it's likely* that you would find workers that are in poverty, needing it as income, finding themselves working obscene numbers of hours. As example, a system that tries to get someone to work 90 hour weeks can easily be worse than no income at all.

*based on the GP, no personal research done

mindcrime2• 13 years ago

So, all of the people making $0.25 / hour doing MT tasks, would be making exactly how much per hour without MT?

I don't agree with any systems that perpetuate poverty.

I, OTOH, agree with any system that doesn't involve initiation of force or fraud, and involves voluntary transactions on a free market basis.

seeingfurther6• 13 years ago

I thought most folks (at least US based) performed the work on the side. Such as receptionists with lots of down time or late night security guards. A lot of this is just busy work. Click this if it's porn... fill in an email from the linked page... categorize this image. Am I totally off base?

pjbrunet• 13 years ago

Check out what Peter Schiff says about the minimum wage. He has some good, reasonable arguments IMO. The minimum wage ultimately hurts everyone. Minimum wage = fewer jobs (quantity and types of jobs) and higher prices.

muhammadatt• 13 years ago

I don't disagree with this sentiment, but I don't think it is correct to say that Mechanical Turk itself is abusive. Mechanical Turk certainly CAN be abused, but it also CAN be used ethically. For example, we regularly use MT to complete tasks that pay workers effective hourly wages in the range of $8-20 per hour, and I know several other requesters who pay similarly.

It is probably most accurate to say that the lack of worker safeguards on MT attracts many users who pay rates that are (IMHO correctly) classified as abusive. Part of the issue with MT is that traditional worker protections like a minimum wage no longer make sense when you have compensation that is paid on a per-task rather than per-hour basis. Under a per-task compensatory system, faster workers will always earn more money (on an hourly basis) than slower workers. For the tasks we do, it is not uncommon for subsets of workers to earn effective hourly rates almost three times as high as others. In many ways this is good - incenting workers to spend their time on tasks that they can complete most effeciently (and away from tasks that they are less suited for) helps promote the most efficient allocation of labor resources.

But how do you ensure that all workers will always be treated fairly when a minimum wage no longer makes sense? What do you replace it with? Its not clear that there is a simple replacement that works in the micro-tasking context.

Our platform (http://houdiniapp.com) helps businesses complete projects on MTurk. One of the things we realized is that many requesters pay low wages simply because they are really bad at estimating how long tasks actually take to complete. We’ve been able to solve this (somewhat) by providing tools that provide real-time feedback on average completion times in order to provide direct visibility into real hourly wages. However we allow users to set their own compensation and some users still end up paying effective wages in the $2-3 (which we do not encourage, but don't have any simple way to prevent).

markhall• 13 years ago

Awesome hack! Brilliant strategy and execution. I'll definitely use this for projects. Thanks for sharing!!!

betadreamer• 13 years ago

the script works great! its very hard to find the email address though. I can't believe people actually dig this for just 3 cents.

austenallred5• 13 years ago

I've mentioned before that I would pay to use this product.

If you had the same tool that worked with a blog search API not only would I pay for it, but you could sell it to SEOs all day long. Just need to find a decent blog search API; I haven't had much luck.

tomgruner8• 13 years ago

It is a good idea for a service, if possible please consider something like https://www.mobileworks.com/ as vincvinc mentioned below that tries to be socially responsible for crowd sourcing instead of mturk.

austenallred5• 13 years ago

At the kind of scale I would use it at I would just hire personal assistants.

That said, I've actually used mechanical turk to earn money in some of my low points (was living in my car working on a startup, out of cash, crazy/long story). I didn't make US minimum wage, but if I were living in Bangladesh, for example, it would have been OK money.

lazyant2• 13 years ago

can you elaborate how would you use an blog search API? like what post/gets are you looking for, what/which blogs etc?

austenallred5• 13 years ago

It could be any range of blogs; it's one of the most effective and least-used means of marketing.

Say I'm a bowtie company that sells to designers; I could look for all the fashion blogs by searching "bowtie" or "necktie," etc. Just generic blog outreach, whether it's little blogs or mommy blogs, would be very useful. Not only do you get some traffic from those blogs, but you get the link juice of some really good links.

It works manually really, really well.

nerfhammer2• 13 years ago

If someone here were to market this product, do you have any suggestions as to how they could reach other customers as well, enough to make it a sustainable business?

austenallred5• 13 years ago

Your target would be SEO. Warriorforum.com, blackhatworld.com, #Seochat, etc.

mzs2• 13 years ago

"The more we can help them out by writing our story for them, the better of (sic) we’ll both be."

I get that, but it sort of depresses me still.

justinwi11• 13 years ago

What part depresses you? That we need to do the write up for reporters?

(fixed the typo...thanks!)

mzs2• 13 years ago

That they are so busy that so often they rehash press releases, that's all. Anyway, it's neat what you did.

dusker2• 13 years ago

How did you guys send emails? SaaS? Your own thing, if so then how did you make sure you didn't end up in 'spam box'?

justinwi11• 13 years ago

Mailchimp. Didn't appear to have a problem with spam - each article was personalized, and included an unsubscribe link.

Could have made it into some folk's spam folders though. No data on that.

dusker2• 13 years ago

good to know, thanks bro

prostoalex3• 13 years ago

Your advice says to enforce embargoes, but the screenshot shows a reporter publishing news early.

Did you not enforce it, and this is a "lesson learned"?

Otherwise, how would a new service know which reporters honor embargoes and which ones don't?

mbrameld• 13 years ago

The caption for that photo says, "We didn’t include an embargo, but we should have…"

seeingfurther6• 13 years ago

Already have an MTurk batch running with 2,250 articles. This is amazing... Thanks!

I set my price at $.05 but only using masters.

Did you use masters?

Did you set multiple assignments per hit or 1 was enough per article?

justinwi11• 13 years ago

I didn't use masters for mine because it was easy enough for me to look at the email addresses that were coming in and see if they were junk.

Would be curious to find out if masters will work for $.05. Please keep me posted! :)

seeingfurther6• 13 years ago

In 2 hours I had 2 HITs completed by masters. Probably going to need to switch to regular workers and closely monitor like you did.

mahranch2• 13 years ago

The masters qualification is handed out to the top tier workers. It's for people who basically use mturk for a full time job and 'usually' output extremely high quality work.

The masters qualification is supposed to open up the higher paying tasks. They will calculate the cost and if they aren't earning, at minimum, .10c per min or better, they will abandon or return your hits. Keep in mind that probably less than 1% of all turkers have that qualification. It's extremely tough to get.

I mean no offense, but requiring a masters qualification for your (being blunt) junk hits is like requiring a PhD to flip burgers at McDonald's.

matthewbaker• 13 years ago

Productize this! I'll pay you today for an easy to use service.

I am absolutely going to give this a shot for our next product release. I love that it simplifies the connection to relevant journalists. It's truly a win-win.

Thank you for sharing!

bledsoestrat• 13 years ago

Definitely should be it's own product. Only snag I found was for double byte characters. Can anyone give me a hand with getting the titles and desc's to pull in properly if the language is in double bytes?

MTurk + Google News API = press

Related Stories