FeedbackArmy.com - simple, cheap usability testing for your site

http://www.feedbackarmy.com/

by raffi • 18 years ago

37 24 18 years ago

feedbackarmy.com

dmix3• 18 years ago

Hmmm purchasing usability testing from a site using a dated design and bright green font on a black background?

No thanks.

joseakle• 18 years ago

Here's my free usability review of that site:

The font is too small.

ivankirigin2• 18 years ago

I thought the same thing. I closed the tab a moment after coming to the page, despite being in the market for such testing.

bayareaguy• 18 years ago

Perhaps they should submit their site to http://www.feedbackarmy.com ? Oh wait...

astrec• 18 years ago

It's not usability testing at all: in a usability test we're interested in how people use the product, not how they say they use the product (although that can be interesting too).

grag• 18 years ago

Anyone know if this uses Amazon Mechanical Turk?

raffi7• 18 years ago

it does (see the questions page)

brandnewlow• 18 years ago

What are the odds that the idea came from reading comments here: http://news.ycombinator.com/item?id=372305

raffi7• 18 years ago

Actually the idea came from http://news.ycombinator.com/item?id=362459

dmix3• 18 years ago

What value to add to the process?

Why wouldn't I just post up the job on Amazon?

iamdave• 18 years ago

This is going to sound incredibly assholish, but it's incredibly true.

FeedbackArmy.com is going to tank. http://www.feedbackarmy.com/feedback.slp

1)That page is why. If someone has the capacity to produce (or have produced) a website, they have the capacity to create a form with two elements and garner feedback. It is incredibly lackluster, easy to replicate and certainly not worth $7 for 10 responses.

2) Based on the comments here people aren't sold on Usability Feedback on websites when the website you've created doesn't suit the needs people are coming to the site for in the first place.

3)The process lends itself quite easily for scammers and cons. There's almost no security measures in place, the design is inherently broken and doesn't provide any peace of mind from the process that you have to perform to get "feedback".

I'm not sold. This can be done about 50 ways better than what it is now; I really do not see myself paying any amount of money for the presentation you've put in front of me. It's an effort, I wont say good effort, I wont say bad effort. It's an effort made, and at this point I'm going to say no to this.

thorax• 18 years ago

It's $7, lighten-up. I'm about to spend $15/person/hour for deeper usability tests and now I can ask less in-depth questions but as often as I want for much less? Pure gold.

aditya• 18 years ago

Wow. There's a lot of negativity and criticism on here, don't be disillusioned, you got something up and out in 2 weeks that says more about you than most other people spending time "commenting" on here :-)

Best of luck, and get on that usability thing these people are talking about...

raffi7• 18 years ago

Thanks for the encouragement. I've been meaning to play with mechanical turk for awhile and this seemed like a good idea. I'm going to hire a professional and get a snappy design going as quick as possible.

patio112• 18 years ago

I went ahead and bought a review. I just spent a few hundred dollars launching a mini-site, what is an extra $7? Rounding error -- if they catch a typo I'll consider it money well spent.

I'll let you know about the quality of results. One request: my results should not be available on a publicly accessible URL. It doesn't bother me, personally, but it will bother some people.

raffi7• 18 years ago

We'll see what happens. I'm watching everyones results like a hawk right now too. You may see a few responses at night (someone who just did a request already has 2) but I find its during the day when they come the quickest.

If you're interested in catching typos you should specify that to the reviewers. The reviewers are good at doing what they are told.

Do you have any suggestions on the publicly accessible URL? I don't want to force people to create usernames/passwords. I think its a hassle for a quick-use service. Or at least its a pet peeve of mine.

How about specifying a password with the request and requiring that password in the URL to access the feedback later on. This way you can subscribe to the results in any feed reader but still have some protection in case people try to guess your URL to see the feedback.

patio112• 18 years ago

Here are the results of my experiment, as promised.

Here's my little mini-site:

http://www.christmasbingocards.com

This is a holiday promotion aimed at my customers. They're overwhelmingly non-technical folks, and not too sophisticated at the Internet. (I am well-aware that most people here would not purchase my software. I'm totally OK with that: you are not the audience.)

I asked the Turkers three questions (and will tell you my rationales):

1) Is it IMMEDIATELY clear what the site offers when you open it? [Rationale: This site is a sledgehammer, not a scalpel. I want them to understand what it offers before they hit their back button. If thought is required, I failed at design.]

2) Try to print out Christmas bingo cards. Tell me how long it takes you from me giving you that instruction to something coming out from your printer. [Rationale: My target is that fourty-something mother with kids, with no special computer skills, can accomplish this in 90 seconds. There is actually a bit of an overlap between Turkers and my market.]

3) If you thought your friends would find Christmas bingo to be a fun activity, would you recommend this site for it to them? Why or why not? [Rationale: The Turking population has a significant overlap with my core audience. Why not ask a quick thumbs up/down?]

What it taught me:

1) The site achieves its singular purpose fairly well. 2) It is attractive enough to move several testers into mentioning it to their friends. (As in, not hypothetical.)

The feedback lets me know "OK, the site is basically OK and needs to be built-out more than rethought from scratch". Knowing that, and the peace of mind from knowing that, are worth $7 to me easily. I hope this service continues getting developed, because I can see a lot of uses for it in my business.

(Yeah, I could roll my own with the Turk API... but I value my time at $100 an hour. This means I never want to be playing the sit-at-computer-refreshing-Amazon-so-I-can-authorize-fourty-cent-payment game.)

sonink• 18 years ago

I tried feedbackarmy - from a purely ROI perspective I think the usability methodology is flawed.

It seems that the folks who give the feedback are actually trying to be a bit too helpful - to the extent that they think about questions as a challenge and try their best to answer them.

For example - If I pose a question that "What does this site do?" in a usbility experiment, I am expecting to figure out if the audience immidiately grasp the utility of a site.

However in this case it seems that the audience takes that as a challenge modifying the question to mean "Can you figure out what this side does?" and try out very hard to find the utility of a site and in the process totally defeating the purpose of a usability review.

It might be an artifact of the way I posed the question, but then I was just following the sites examples. I think using the site in the way it is setup (messaging, mturk) the concept does not give a good ROI w.r.t usability.

However, one thing that I did like was that the turkers gave some suggestions to improve the site - some of which was really valuable. So maybe if you change your positioning to something like an active feedback/testing tool it might work better.

FeedbackArmy.com - simple, cheap usability testing for your site

Related Stories