Discovered: Botnet Costing Display Advertisers over $6,000,000 per Month

http://www.spider.io/blog/2013/03/chameleon-botnet/

by blahpro • 13 years ago

128 87 13 years ago

fein2• 13 years ago

Well, that's what you get for being in the click based ads game really. At this point, I would assume that these companies should just accept this as an occupational hazard. It's not like they can ever really beat the bots.

pfarrell2• 13 years ago

At Strata (O'Reilly's BigData conference), there was an entire session track dedicated to fraud prevention and legal movement on this type of stuff. Although, you will never be able to stop this (unless you stop people from being driven by money) the data community is taking this kind of detection very seriously. Every business, if it get large enough, is going to care about this type of fraud, if only to pay it some amount of lip service

I saw a presentation. From a data scientist at bitly. She showed that spammer links have a distinctive traffic shape (constant over time) while real links have a totally different one (initial peak followed by logarithmic drop to zero). Similar patterns exist in advertising campaigns. 6MM/month seems like a lot of breakage that someone is fine cutting that check.

adyus• 13 years ago

What's to stop a botnet operator from incorporating that data scientist's findings into their bot behavior? It's a chicken & egg problem, and it will hopefully lead to a better alternative to online advertising (perhaps something more user-centric).

elmuchoprez2• 13 years ago

"Well, that's what you get for being in the click based ads game really."

Do you find something particularly sinister or unethical about click-based ads (more so than any advertising)?

fein2• 13 years ago

Oh no, not at all. I realize that could have come across as malignant, but it was just supposed to be a neutral assessment.

I do freelance dev work for some guys that run ads; it's just another business to me.

tatsuke95• 13 years ago

I don't.

But this is analogous to TV advertisers complaining about TiVO and the ability to skip commercials. Can't stop it!

elliottcarlson• 13 years ago

No matter what line of work you are in, there is a cost of doing business related to fraud - that does not mean it doesn't require some kind of attention even when you can't combat 100% of the fraud.

binarymax• 13 years ago

Very informative and, though it is not explicitly stated, we can infer that this evidence cuts to the point of how low some competitors will stoop to exploit pitfalls of web advertising.

The team at spider.io has found a great niche and has impressive results - I always enjoy seeing posts like these pop up from them. Keep up the great work!

readme• 13 years ago

The infections by state chart would be more interesting if it was per capita.

d23• 13 years ago

I find this fascinating. How does chameleon infect its victims? Anyone have further reading? Botnets seem incredibly interesting.

unix-dude• 13 years ago

Botnets dont really infect their victims. A botnet is just a network of compromised computers (Bots).

The malware that forces your computer to participate in the botnet can be delivered by any avenue imaginable. Drive-By Downloads, crapware, embedded into pirated software, etc. Not sure how chameleon specifically did it.

andreasklinger2• 13 years ago

Is the comparison valid to say botnets are the bacteria of the Internet?

JonnieCache• 13 years ago

Botnets are inherently malicious. This is obviously not true of bacteria. But in a way they are similar I suppose. I don't think its a very fruitful analogy either way.

andreasklinger2• 13 years ago

I am not sure if can be called non-malicious but there is definitely a new breed of botnets coming up

e.g. http://internetcensus2012.bitbucket.org/

bd_at_rivenhill• 13 years ago

More like the parasites of the internet.

lucb1e2• 13 years ago

Except parasites have no central control or a will to do anything beyond survival.

cletus• 13 years ago

As much as the HN crowd might rail against walled gardens (most notably iOS) and managed platforms like ChromeOS, every time I read one of these posts I think that for the vast majority of people, it's the best thing for them.

Botnets typically don't spread in a sophisticated way. Most of the time it's spam emails or dodgy ads with "hey! install this random .exe file and you can have emoticons in Outlook!"

I think Chrome has shown us the advantages of an automatically updated browser. The future in personal computing I think lies squarely in an automatically updated (even managed) sandboxed environment.

This isn't to say that's right for everyone of course.

But how much fraud, extortion, DDoSing, identity theft, invasion of privacy (eg ratting), etc do people need to put up with before they demand a better way?

EDIT: to address two points:

1. Side-loading is orthogonal to the issue of a sandboxed managed environment. I agree users should be able to side-load. Most won't know how and won't care and that's a Good Thing [tm];

2. Sure the central server can get compromised but the thing is botnets rarely spread in a sophisticated fashion. It's all social. The Facebooks, Apples and Googles of the world have far more experience and a far better track record in dealing with these kinds of threats.

sc0rb3• 13 years ago

I don't think many home users are using Outlook.

nmcfarl• 13 years ago

Until she got an iPad my mother ran Outlook. It’s all she knew how to operate for a mail program, and she "wasn’t learning something new".

I don’t think she was alone - I’d bet a ridiculous number of $300 netbooks have a $100 Windows OS and $200 MS Office Suite installed on top.

muyuu• 13 years ago

Seriously?

eel• 13 years ago

Strictly speaking, you are probably right. Outlook has often been bundled with the higher tiers of the Microsoft Office suite and omitted from the lower tiers. But Outlook Express and Windows Mail? Certainly there are many home users using them. They've been bundled with Windows or IE for years.

I don't think this distinction affects the grandparent comment's point.

cliffu2• 13 years ago

Sandboxing is great for the typical user.

But we should never stop demanding that we can install our own certificate roots for signed binaries to verify against, and that users are allowed to make their own decisions about allowing access outside the box (if they think they are capable).

Retric2• 13 years ago

Sandboxing only helps so much. You can run a browser extension in a sandbox that defrauds advertisers.

StavrosK• 13 years ago

Honestly, this should be requirement #1 in every system. Give the user an easy (but not easily exploitable, of course) way to unlock the system.

martinced• 13 years ago

"The future in personal computing I think lies squarely in an automatically updated (even managed) sandboxed environment."

Until the day where the main server serving the automated updates gets compromised and instead of serving, say, an updated Chrome, it serves a version of Chrome which a) is compromised on the behalf of a botnet master and b) never ever accepts any other update automatically.

Because people have been trained never to update their browser themselves anymore they'll think everything is fine.

Because hundreds of millions --if not billions-- of people are running Chrome suddenly you have the biggest botnet ever out there.

I find it very interesting that you fear extortion, exploits, DDoS, identity theft, invasion of privacy and whatnots as an argument for putting something in place which potentially can be way more destructive.

But of course this shall never happen right? Just as we haven't seen FaceBook getting penetrated and just as we've seen any major bank getting hacked right? And rogue employees also don't exist right?

Be careful about what you wish in the future...

hyperbovine• 13 years ago

Did the "centralized anything == bad" camp show up or something? This is a surprisingly hyperbolic rant about browser security.

seabee• 13 years ago

How is this different to the status quo of hundreds of millions running old exploitable versions of $BROWSER, infected with malware which hijacks your internet connection? All you've done is swap one delivery mechanism for another.

You don't fix this by switching out automatic updates for manual ones, because users will blindly install an update even if it's a badly-designed 'update dialog' popup on a shady website. You fix it by ensuring users can trust their software, and care enough to do so.

nemothekid• 13 years ago

It is really just choosing between lesser of 2 evils.

1.) A million people using $BROWSER, never updating, continually downloading free_ipad.exe

2.) A million people being owned by an intelligent hacker because of a fault of Google, that is relatively quickly patched, and I'm sure someone will figure out something to disable those rouge Chrome installs.

The first scenario is much, much more likely to happen, and while the second could happen I doubt, Google would sit on their hands while it does happen.

I guess the only problem is in the second issue, its not your fault if you got screwed. Its your mom's, and just your mom's, problem if she downloads free_ipad.exe, but if Google is hacked all the sudden all your info is comprised and it wasn't your fault.

Considering these two options however, IMO, I'd rather place my faith in Google overloads keeping us all safe.

diminish• 13 years ago

>The future in personal computing I think lies squarely in an automatically updated sandboxed environment

- When the future moves to sandboxed environments, (financial) risk will move there too and attacks will target that future in the sandbox, and attacks will develop with technologies allowed within that sandbox. Disallowing anything but port 80/443, appeared to be a secure environment 10yrs ago, and now javascript, cookies, and the web sites incur many more diverse types of attacks than earlier, within what was allowd.

- A managed, self updated environment, or a walled garden, already gives the authority a bigger power on every activity and that's often a bigger cut (no botnet can dream of 30% of all transactions...) and a draconian power than sporadic illegal attacks to some business. And this is a bigger risk overall, to diverse businesses, than this poor botnet poses.

trotsky• 13 years ago

I don't disagree with your conclusions about managed platforms being better for enduser security, but for what it's worth a vast majority of botnet payloads are being loaded by drive by exploits, email based infections are generally rather targeted these days because spray and pray email infections typically get caught in spam filters.

While the bulk of the drive bys come from plugins that don't exist on your classic walled garden (ios) webkit is far from invulnerable. ios (and others) are quite resistant, but so was ASLR at one point - malware rises to the occasion.

cs702• 13 years ago

cletus: I agree, but also think that lack of diversity in user environments is a big contributing factor to this problem. With 90%+ of people using only three computing platforms (Windows, Android, iOS), criminals and other sociopaths have a powerful economic incentive to develop antisocial viruses and botnets. So long as only two or three user environments remain dominant, people will be exposed to this kind of 'Irish potato famine' risk.[1]

[1] http://en.wikipedia.org/wiki/Great_Famine_%28Ireland%29#Pota...

saraid216• 13 years ago

This is a great point and you should totally start a competitor.

coopdog• 13 years ago

Totally agree, and I think part of the popularity of this whole 'web application' swing is because browsers like Chrome et al are so well sandboxed also. It makes it almost entirely risk free to automatically download and run random code (javascript), something that would be entirely unthinkable if done natively in today's environments.

belorn• 13 years ago

The case about walled gardesn is the same old argument but in new clothes.

Every so often, someone comes along and says that people don't know what is good for them, so its better that someone decide for them rather to let "them" decide.

Sometimes it happens in politics, and we get dictatorships. Sometimes it happen in the market, and we get illegal monopolies. Sometimes it happen with the police force, and we get a police state. Sometimes it happens in crime, and we get mafias. Sometimes it happens in families, and we get forced marriages. And sometimes it happen with software, and we get walled gardens.

Deciding for people by removing all choices are not "a better way". We just tend to forget what the cost are of losing the ability to choice for our self. Its that, or we are just saying that all software around us is just of such small importance that being allowed to decide your own fate is not an important thing.

JVIDEL• 13 years ago

Indeed its a matter of basic education really: most people don't know about spammers or how well organized they are but the moment they learn about their tactics they catch on quick and stop falling for link bait blogs and comment spam.

Better that than not being able to install what you want because "the garden protects us!"

jpdoctor• 13 years ago

Using the assumption that there is never just one cockroach, what is a good multiplier to arrive at total-fraudulent dollars-per-month?

Nursie4• 13 years ago

A few things -

1. Why?

2. How widespread is this in general? How long before most web advertising is bot-fraud as users learn about ad-blockers?

3. Didn't realise my mouse traces were being recorded by advertisers in such detail.... I do not like this.

elmuchoprez2• 13 years ago

1.) The only "useful" application of this that I can immediately think of is to drive up the cost per impression on your competitors. But this seems like a very short term strategy as anyone doing meaningful online marketing is watching the cost per acquisition (a person who actually buys something) just as close or closer than impression cost, and your acquisition cost is going to go through the roof if you get a bot attack.

2.) I haven't been able to find meaningful info on this yet.

3.) I work in e-commerce tech. I had NO IDEA how bad it was before I got into this industry. Just had a sales call with a company who is tracking 400 unique data points per second on users. They track mouse clicks, mouse movements, what your highlight, how long on site, navigation pattern, buying behavior form other in-network sites, page position, etc... and then use that data in real time to dynamically generate promos and offers to "encourage buying behavior".

Nursie4• 13 years ago

That there is quite scary! May have to investigate "NoScript" a bit more closely...

jdalgetty3• 13 years ago

money, that's all.

Nursie4• 13 years ago

Oh sure, I understand that motivator.

I was just wondering who it benefitted. I guess the site operators themselves, but it seems pretty underhand.

andyakb• 13 years ago

They own the sites that are displaying those ads and get a share of everything google makes, so their adsense earnings go through the roof

slig• 13 years ago

I don't think they can fool Google. In the article there's patterns of clickthrough and mouseovers for both a real user and a bot. If anyone, I believe Google can detect what's likely to be a fake impression and click. It's their bread and butter.

wcfields• 13 years ago

1. There's a link a comment up in the thread[1] that points out Ghost Sites: Websites with high visitor traffic but obviously no one would ever really visit. Like Babypowder.net or some such malarkey. If you're running a ghost site, you contract out to a botnet to deliver clicks and impressions at rate lower than the return from the advertising.

2. The article shows a bunch of these ghost sites so I would suspect that it's common in the world of scamming/ghost sites to drive up their CPM (Cost per Thousand). Hell, if I was running a legit site, I'd outsource the bullshit ghost sites to a Russian company so that my own CPM gets higher. As an example: I bet the value of "Baby Powder" as a search term is probably twice what it should be.

3. Get "Live HTTP Headers" or run your traffic through a proxy and see how many HTTP requests you generate just using a basic ad. I worked in Rich-Media Advertising and the way we tracked was every interaction. Right when I left in 2008-2009 we started tracking more and more. Really depended what the client bought/paid for. But we'd always track opens, clicks, etc.. the basic stuff. We didn't do heatmaps or X,Y coordinate tracking, but we had the data.

It was impressive to see, we would host our units on Akamai and due to the sheer traffic were unable to do real-time SQL inserts, so we would crunch the HTTP logs in hour increments after the fact to generate the data. So a typical GET string would be ad.domain.tld/?id=adunitMD5&type=Click&x=0&y=0&action=MD5

Of course it wouldn't be that verbose, but the GET string would be quite long with all the data. This was all done through a combo of JS and ActionScript.

[1] http://www.adweek.com/news/technology/meet-most-suspect-publ...

rsingel• 13 years ago

Well that leaves the real questions: what sites were the bots clicking ads on, who owns those sites and which ad networks were they using?

megablast• 13 years ago

> spider.io has observed the Chameleon botnet targeting a cluster of at least 202 websites. 14 billion ad impressions are served across these 202 websites per month. The botnet accounts for at least 9 billion of these ad impressions.

The odd thing is that either these sites are already very big, or there are others ways they are getting 5 billion ad impressions.

A list of these 202 websites would be informative. I guess a number of them could be fake, to throw the scent off?

oroup• 13 years ago

Perhaps a coincidence, but this AdWeek report[1] flags a number of "ghost sites" that offer huge volume of impressions for sale on the exchanges and have real advertisers but don't seem to have any actual human beings present. One site in particular they mention usbuildingdigest.com has a very large number of DSP and other data integrations[2]. Suggestive to say the least. A number of other sites are mentioned in the AdWeek report.

[1] http://www.adweek.com/news/technology/meet-most-suspect-publ... [2] http://imgur.com/WHxmmHo

kreilly• 13 years ago

This list of sites will change in 60 days. This sites are like pop-up store fronts for ads.

epmatsw2• 13 years ago

It's interesting that the infection seems most common in the Southwest. Do botnets like this spread geographically based on email address or physical connection/proximity? Or are the targeted sites or infection points targeted at users in the southwest? Or are people in the north/northeast more likely to use anti-virus software or be savvy enough to avoid this?

claudius• 13 years ago

They appear to have coloured states based on the number of infected hosts in such states. Since states in the northeast are generally smaller (i.e. less populous) than, say, California, we can assume that there are also fewer people getting infected there.

E.g., there are approx. 1e6 people in Maine and 4e7 people in California. If you assume 1e2 infected hosts in Maine (0-99) and 3e4 in California (>1e4, 1.2e5 in total), you get an infection rate in California of about 7.5 that in Maine.

Given the very coarse graining in the data source, such a factor can either be dismissed as statistical fluctuation or you could try to explain it using, for example, an infection model that favours geographical proximity, such as one based on Facebook friends. Furthermore, it might well be that internet connectivity is better in California than it is in Maine and the bot prefers hosts with high uplink rates. I don’t know :)

Edit: We don’t know what websites were targeted, but maybe they ran ads that would prefer users from the southwest for some reason?

epmatsw2• 13 years ago

Ah, didn't realize they weren't weighted. Still, it'd be interesting to see if the apparent bias to the southwest is statistically significant.

entropyneur• 13 years ago

Wow, I didn't realize the arms race has reached such heights already. Looks the bad guys are bound to win eventually.

smtddr• 13 years ago

There are some crazy clever schemes out there for click fraud. Check out this link - _WARNING NFSW IMAGES_ http://www.behind-the-enemy-lines.com/2011/03/uncovering-adv...

This was estimated to be making $500K a month before being found... and was a work of pure genius.

gingerlime2• 13 years ago

Interesting read.

I'm still not sure I understood the role of the "HGTV" sites and how the fraudster was getting money by showing the HGTV ads (even after reading the comments on the post explaining this). Weren't the ads on those parked domains enough to generate the revenue for the fraudster?

brador• 13 years ago

Maybe it really is just windows users running IE 9 on windows 7...and maybe it just crashes on clicks sometimes because the tracking overloads it?

Do they have the bot code? I didn't see anything about where it came from...just an assumed analysis of effect. Just saying, it might not be a bot or malware at all.

bcherry• 13 years ago

The analysis of click and mouse traces location distribution vs humans at the end makes it pretty clear that these are not humans.

jdalgetty3• 13 years ago

Any idea what the list of 200 sites are?

arbuge• 13 years ago

The only guaranteed antidote to this kind of fraud is performance advertising (pay per sale). I think pay per click and pay per impression, though arguably useful for brand advertising, will always be vulnerable to sophisticated scams like this.

lutusp3• 13 years ago

> The only guaranteed antidote to this kind of fraud is performance advertising (pay per sale).

It's guaranteed, but it probably won't work: "Thanks for doing business with us -- by the way, where did you hear about us?" "Online, but I forget where."

Advertisers need to be able to associate an advertisement with a result. Otherwise the effectiveness of advertising is a myth.

FollowSteph3• 13 years ago

All the scammers have to do is purchase say x% of clicks and the. Refund the money. It's very hard to track which ad click generated exactly which sale. For example through Adwords content network you don't know which site generated the click/purchase. You can know that content network ads gets more refunds, but without access to googles data you can't know the specifics.

So as good as CPA sounds, its not full proof either. You can just bypass it with either stolen credit card purchases or refunds.

skore• 13 years ago

That does not address what the post you're replying to stated: That PPS is a more trustworthy metric than PPC. The user is rarely, if ever, asked in this - it's all tracking.

guelo• 13 years ago

That's not how it works. Typically you have a referer ID in the URL which tracks the source.

jyu• 13 years ago

PPA can be as misleading as CPC or CPM. If you ask for an email, it could be iframed with a more compelling offer. If you ask for a credit card purchase, it could be done with big blocks of stolen credit card numbers. This also does not include covariance between marketing channels aka the attribution model problem (ex. if you blast TV ads for freecreditreport.com then the CPA for online advertising magically drops).

badfortrains• 13 years ago

There is no guarantee against this, which is part of what keeps web advertising prices low. Fraud is essentially priced into the product. The title is misleading, as it assumes all cost is bared by the advertisers. In truth part of the cost is implicitly payed for by legitimate website owners/the ad network who must price impressions lower to offset this kind of fraud

von_tenia• 13 years ago

I wonder if Ad platform companies like adwords will see a drop in their revenues once the Botnet will be dismantled...

lucb1e2• 13 years ago

Or perhaps an increase because ads become worth more. Fake clicks make less sales, so if there are more sales, people can spend more on advertising. Of course it needs to be at a very large scale for it to influence the actual price per advertisement, but this botnet seems to be rather large-scale.

gluejar• 13 years ago

I'm wondering if this might be related to the twitter spam discussed at https://news.ycombinator.com/item?id=5373161

chwolfe• 13 years ago

Before you blacklist the IPs listed in the article, it might be worthwhile to query your transactional history and verify real purchases are not occurring on those addresses.

When I did this, a few of the IPs had a significant number of orders. Interestingly, the IP with the most orders mapped to E! corporate headquarters.

http://www.networksolutions.com/whois/results.jsp?ip=208.78....

kingkool68• 13 years ago

I'm fairly confident all of the revenue from one of my sites comes from botnet ad clicks. I use CloudFlare and when I set it up at first I used the standard settings for blocking bots. My ad revenue flat lined. Took the botnet protections off and my ad revenue went right back to what it was before.

GhotiFish• 13 years ago

haha, I like this post.

Are you gonna leave it? I must admit, screwing advertisers doesn't feel so wrong, I'm certainly no fan, but you can't ignore that it's not very ethical.

Given that it's not technically your fault, you'd very likely never get blamed, and your actions will likely not change the world in any way, what will you do?

Discovered: Botnet Costing Display Advertisers over $6,000,000 per Month

Related Stories