AVG disguises fake traffic as IE6

http://www.theregister.co.uk/2008/06/26/avg_disguises_fake_traffic_as_ie6/

by nickb • 18 years ago

33 22 18 years ago

theregister.co.uk

gm• 18 years ago

Any opinions on the best way to deal with this?

ajross• 18 years ago

There aren't any. The User-Agent header has always lied, at least a little bit. What AVG wants from your site is the same thing that the users do: the content (in this case, so that they can scan it). If you insist on giving them a different page (like an error page) than a user would see, they're going to take steps to prevent you from doing that.

Remember that the desired outcome here is to see fewer AVG hits, not to "screen them out" just because they're not a human being (although they are a necessary condition for a human hit coming later). Try reducing the unique URL count on your site (basically: make it more RESTful and eliminate all the query strings on broadly identical content) so they have fewer URLs to crawl. Set your caching headers properly so that they don't have an excuse to recrawl you too often. That won't force them to hit you less, but it will "help them to help you" by reducing the total bandwidth. Remember they don't want to pull any more than they have to either.

BrandonM• 18 years ago

What AVG wants from your site is the same thing that the users do

Then shouldn't AVG report the same User-Agent that the user's browser is reporting?

avalanche2• 18 years ago

I think the solution should be fairly simple. Since AVG is not a real browser, despite telling a web server that it is, it should be easy to craft a piece of Javascript and cookie on a webpage that would return a proper result in a real browser like IE, Firefox, Safari, or Opera. If a proper result is not returned, then the web server can just do a 302 meta redirect to AVG's website as mentioned earlier. If implemented widely on web servers (say as an apache module or PHP patch) this, in effect, would cause AVG to become it's very own DDoS attacker. I don't think they would want this, but it'll probably happen if they don't re-think the way they do things.

mleonhard• 18 years ago

How would that work? The AVG robot probably only fetches the page's URL. By the time you serve up a JavaScript, you've already lost the ability to redirect the robot.

Also, once enough people start redirecting the AVG robot to the AVG website, they'll just update the robot to ignore those redirects. It's antivirus software so it gets updated very often.

revicon• 18 years ago

Google / Yahoo do this as well to check up on sites to make sure whats served to their bots is the same as whats served to real users. Google has 3 or 4 different fake user agent strings they use for the googlebot

jonknee• 18 years ago

Google doing it isn't nearly the same as 10 million users doing it. In fact, this type of activity should be limited to search engines, not client-side apps.

Hexstream• 18 years ago

User-Agent information is inherently unreliable. I don't see why people are upset when a misguided practice that already provides false results with an unknown margin of error gets even worse. If they insist on using such a metric then it's their responsibility to assume the risks that it incurs.

petercooper2• 18 years ago

One wonders if there's an "advantage" you could gain from this.. If your site is getting requested by AVG but NOT a valid browser, you can assume someone did a search that you appeared on but they did NOT click on you. This metric could be useful for improving your CTR in regular search results, something totally impossible to measure before.

spydez• 18 years ago

The problem being you've no idea why they did not click on you - only that they did. Thus, you can know something's wrong, but not why.

And how are you going to improve if you don't know what's wrong?

petercooper2• 18 years ago

In SEO that's a pretty common problem.

Hoff• 18 years ago

The bad guys know how to bypass this check.

It took seconds to figure out a way that could cloak malware against this AVG check.

Malware is a problem.

This is not a solution.

This is another botnet.

Xichekolas2• 18 years ago

Sounds like there is an easy solution to this problem... just block all IE6 users... kills two birds with one stone!

</tongue_in_cheek>

Stabback2• 18 years ago

I'm actually building a web app with a small start up, and we just do not have the time to make our service ie6 compatible. We are going to let users know that and redirect them to a very plane jane site where you can still get the information but not very efficiently. However our market is mainly tech savvy people so it should work out.

mdemare• 18 years ago

Fair enough, but sometimes tech savvy people use computers of tech unsavvy people (friends and relatives, internet cafés, corporations with reactionary IT departments...)

Xichekolas2• 18 years ago

Just use the js widget on http://www.savethedevelopers.org/ ... no sense writing your own.

I'm also ignoring IE6, crappy old corporate computers be damned. It's just not worth my sanity.

ideamonk• 18 years ago

AVG is an antivirus. When users install it, they give it full privileges. A program like AVG Antivirus runs on many computers. Ofcourse this gives AVG enormous power to affect ad market, not only will CTRs go down, but also if AVG is corrupt, they can make auto clickers or have rival's ad income down by a great extent, or eat up adsense impressions of rivals. I wonder why didn't anyone think before implementing this.

AVG disguises fake traffic as IE6

Related Stories