johnli

Shameless plug for the service we built to solve this for ourselves - https://www.pickfu.com

The MVP was to meant to replace asking randos at a coffee shop about your idea/design/etc. Over time we've added demographic targeting, follow-up questions, and other requested features. Nowadays online sellers and other creators use it like an online focus group to validate ideas and products.

Some other ideas for finding an audience for your MVP:

  - google the exact problem you're trying to solve, reach out to authors/bloggers who've written about it
  - online ads to your target demographic 
  - LinkedIn prospecting (if you have a target customer profile)
  - Subreddits of your target problem area
  - Start writing/tweeting about the problem space and engage the audience that follows
  - comments section of relevant YouTube videos

As noted before, it's tough to know that 100% of the responses are truthful on any survey. With PickFu, you can read the explanations to get a feel as to which respondents put some thought in.

I'm sorry you had a rough experience with that specific music-listening job - we've never had a more time-consuming question submitted. Most are in the order of "which site do you prefer?", like this current one: http://pickfu.com/AGSIBT. These get answered far faster by the MTurk workers.

Ah yeah, I get what you're talking about. It's one of the features I've been wanting to add for a while too - filtering the responses by any demographic combination. Not pri 1, maybe in the next version. =)

Thanks Lukas, I plan on being there. Looking forward to meeting up - you guys are doing some awesome things with MTurk.

Actually, there is a way you can view the votes by (gender,age-group,etc.) - click on the "Option A vs. Option B" tab right under "Who responded to your question?" Is that what you were looking for? If so, it's our fault for not making it more visible.

Thanks for the heads-up about the mturk job search - we're making tweaks so that those previews show a sample question from now on. Helps keeps the responses more random too.

Thanks for sharing it!

It would require more effort to get 50 responses from a tiny demographic segment than a large one, so I was wondering if having the price inversely correlated to the segment size was reasonable to an end user.

Hmm, would you pay more than twice as much to target a demographic segment that was less that 50% of the population?

We looked at using Qualifications a bit, but quickly decided that it wasn't worth our time to deal with them, at least for PickFu in its current state.

Our respondents don't get paid unless they actually explain their answers - which both reduces false responses and provides unique value and insight to the service.

For example, we used this feature to dogfood our own FAQ page: http://pickfu.com/4UQM7U

Currently no, you can't set up a higher # of responses.

We found that 50 responses offers a good set of opinions without taking too long to finish. What are some other options you'd like to see?

another one of the creators here, thanks for the warm comments.

a follow-up question - how much more compelling would this be if you could choose/influence the demographics and # of responses (potentially at a higher cost)?