GoogleBot as QA tester

http://statsheet.com/blog/meet-statsheets-lead-qa-tester-googlebot/

by johnbb • 17 years ago

28 14 17 years ago

fizx2• 17 years ago

I always "wget -r --header 'Cookie: login-credentials-if-neccessary' | grep 500" my staging site before sending it to production.

I also use hoptoadapp.com for error reporting.

fizx2• 17 years ago

Edit: For clarification, this isn't the only thing I do, but it's a helpful sanity check.

ggrot• 17 years ago

Checking for 500's seems like a pretty minimal test suite. There are plenty of other errors that could occur.

RobbieStats5• 17 years ago

Sure, but the point is that when resources are very low you may need to get by with the minimal amount of testing required.

The options are either to spend a bunch of time trying to squash every potential bug, or know that some bugs will get through but at least users will benefit from having a bunch of new features to play with.

WALoeIII• 17 years ago

This only ensure the pages don't explode, you still have no idea that you are showing NASCAR stats on your NBA homepage...

Crawl and Fuzz locally:

http://github.com/relevance/tarantula

RobbieStats5• 17 years ago

Actually I do because I'm the one that developed those pages. Why would I put NASCAR stats on a NBA page? ;-)

Also, as a last resort users will tell me if something is off.

PhilChristensen• 17 years ago

I think you should reconsider those statements.

Just because you wrote it doesn't mean it's doing what you think it is.

And users really can not be counted on to report errors. Even if they do, the average bug report reads something like "So I put in your address into the google, but when i got there nothing came up! I tried over and over, but every time I clicked on the google, nothing happened! I asked my brother's boss' kid, who's a real whiz with computers, and he told me I should 'eat my cookies' or something, but I'm lactose intolerant!!"

dandelany• 17 years ago

PEOPLE. downvote != disagree.

nathanb• 17 years ago

I agree that this method might work for statsheet.com, at least in its current incarnation, but it's a bit of a special case. Just because you can rely on spot-checking and only reacting to critical errors when you're strapped for resources doesn't mean it's a good model.

It's like writing a post talking about how you've eaten beans and rice for the past four months. Maybe you needed to do it that way to keep your company afloat, but that doesn't mean that the post constitutes advice.

When bootstrapping a startup a lot of exceptional circumstances occur. Not having resources to test a massive web of pages and data is one of them. The last paragraph of the post makes this clear.

NathanKP• 17 years ago

Just because you can rely on spot-checking and only reacting to critical errors when you're strapped for resources doesn't mean it's a good model.

I agree. The proposed system can track critical errors and syntax errors, but it can't really keep track of logic errors. The stats may not cause any errors but Googlebot can't tell if they are accurate or not.

RobbieStats5• 17 years ago

Why doesn't it constitute advice? There are lots of entrepreneurs that are strapped for resources. I don't find this as an "exceptional circumstance". My point was simply that it is ok if you don't have 100% test coverage or any automated test coverage. I think that is a waste of resources in a very small startup in many cases.

GoogleBot as QA tester

Related Stories