Should I block all of China from my online store?

https://news.ycombinator.com/item?id=3115042

by bsenftner • 15 years ago

2 3 15 years ago

news.ycombinator.com

So, I have a not-really-in-beta-yet online store I'm developing in Drupal, aimed at 3D animation geeks. I've been a 3D developer since '85, and have some pretty advanced stuff going into my online store (see www.flixor.com, my technology company.) Anyway, perhaps because I globally patented my more advanced technology, companies in China are constantly crawling both my not-really-open online store and my technology company's site, consuming some significant bandwidth. But more important: their constant traffic impacts my work getting the store open.

I've also noticed that every variation of my technology company's name has domain squatters in China. Plus my patent in China receives challenges, but I've survived them all so far.

At this point, as I get closer to launching my store, their crawling and seeking access to my backend is getting out of hand. Last weekend an IP address in China consumed all public data from the site. I can, through my Drupal backend, simply ban all of China's IP range.

Would that do any good? Any suggestions?

This looks promising: http://www.parkansky.com/china.htm

epc2• 15 years ago

Anything you do is going to impact performance on your site. It sucks, I'm sorry.

If you go the .htaccess route, every single new connection will get pattern matched against the _deny from_ patterns. It's a bit better using CIDR notation, but still, every connection will take a slight hit as the pattern match is run. I assume that Drupal does a similar pattern matching scheme to block/permit access.

A determined adversary will realize that you’re blocking by IP addresses and resort to something like TOR or a VPN to crawl your site.

Another option is to use something like IPTables with some intelligence on your server site to dynamically block anyone slurping lots of data off the site (but be certain to whitelist Google/Bing/other "friendly" crawlers).

You could try something slightly bizarre: on detecting adversarial traffic from what you suspect is a Chinese client, return the complete works of the Dalai Lama, a history of tibet, possibly treatises on Falun Gong or the true history of Tiananmen square, and put the Great Firewall to use for you. Depending on the ISP and the day’s censorship regime, you might get the GFW to block your site from access from behind the GFW. You don't care, you don't want the traffic anyway, and from the GFW's perspective your site will be marked as a "dangerous" site with inappropriate content for consumption in China.

bsenftner• 15 years ago

Brilliant, I love the idea of sending them the true history of Tiananmen square!

epc2• 15 years ago

If you can, look at reactive responses, rather than proactively blocking everything (and thus impacting site performance for all potential customers). So something in drupal or mod_security which watches inbound traffic and if a pattern emerges, then respond until the traffic dissipates.

Another thing might be to require a login for the portions of your site where you want to protect the content, or which are dynamic and for whatever reasons caching isn't helping mitigate the load. Again, determined adversaries will just add the login to their scripts and carry on, but you can now block by the authentication rather than by IP address.