ebol4

This article is very poorly written, and many of the points made in it are unfounded or irrelevant. I completely support abolishing the practice of tipping, but using bullshit arguments like "it's un-American" and "Economists don't get it" don't lend any credence to your stance on the issue. The only point that directly cited actual data was the first point. This article shouldn't be voted up as much as it is.

Don't they just have to not be parallel?

There are a LOT more file-extensions that you'll want to ignore.

It'd probably be a good idea just to whitelist filetypes that you actually want to crawl, rather than trying to blacklist all the ones you don't.