Researchers Expose Cunning Online Tracking Service That Can’t Be Dodged

http://www.wired.com/epicenter/2011/07/undeletable-cookie/

by bmcmanus • 15 years ago

191 146 15 years ago

_delirium• 15 years ago

The article focuses mostly on legal measures (e.g. lawsuits, regulation), but my guess is that those would only deter the largest companies. What I'm more worried about is why 'incognito' modes in current browsers don't appear to stymie this tracking, and how likely it is that that can be fixed.

mtogo• 15 years ago

From what i gather it's basically just an evercookie. Block kissmetrics with a host file, firewall, Ghostery (Not the chrome version, though), RequestPolicy, etc or defeat evercookie through usual means and you'll be fine.

gojomo2• 15 years ago

I, too, would be rather surprised that incognito/private-browsing would share cached data (and thus ETags as sent on If-None-Match requests) with normal browsing.

Looking at the researchers' paper...

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1898390

...it's not clear that's what they're claiming. One quote is that "Even in private browsing mode, ETags can track the user during a browser session."

That suggests they may be concerned about cross-site tracking within a single private session, and the possible expectation that 'private browsing' prevents tracking from site to site. (I've never had that expectation; only that a private session is not connected to distinct prior private and non-private sessions.)

catch23• 15 years ago

I just tested incognito, it seems to defeat the etags mechanism of tracking.

cleverjake• 15 years ago

is there any indication on where the data is stored?

thirsteh• 15 years ago

Haven't looked at KISSmetrics, but I assume it's some manifestation of evercookie: http://samy.pl/evercookie/

benregenspan• 15 years ago

Yep the "never before seen in the wild" ETag approach was implemented September 2010 in an easy-to-use library. Very clever, but it seems like Samy Kamkar deserves the credit here, not the brilliant researchers who found the library's approaches being used somewhere.

Joakal2• 15 years ago

Yes, but not easily seen unless you have a tool like FireBug to see.

It works by setting a unique cache tag (etag as in screenshot) for each user of a resource such as HTML, JPG, GIF, etc files. The later requests can then be extrapolated of what the user views per site. It's in effect, a cookie.

I think it's quite brilliant as an alternative to cookies but unfortunately I can't use it as a form of cookies as they are not a HTTP standard.

More: https://secure.wikimedia.org/wikipedia/en/wiki/HTTP_ETag

justincormack5• 15 years ago

Etags are an http standard.

Joakal2• 15 years ago

Can it be dodged by emptying browser cache as well blocking iframes which I assume is causing such content to be stored in browser?

Edit: seems so: snip ... the persistent tracking can only be avoided by erasing the browser cache between visits.

achille• 15 years ago

Looks to be using the individual etags associated with each cached object. Pastebin: http://pastebin.com/FhUYuRsb

trotsky• 15 years ago

That Can't Be Dodged

Very interesting article, but the proclamation you can't avoid it seems a bit too far. When my browser exits it both deletes cookies and clears the cache, which looks like it's enough to break the tracks.

chubot• 15 years ago

I do that too, but I don't think it's enough.

I use FlashBlock, which I think is enough, because they're apparently using flash cookies to recreate regular HTTP cookies (or something like that).

Flash is a huge POS in so many ways.

jacques_chester2• 15 years ago

There's a whole bunch of places to stash unique identifiers. And you only need to overlook one of them, because they will repopulate all of them the next time you hit a KM-using site.

HTTP cookies, flash cookies, ETags, HTML5 databases ... it just goes on and on.

Tobu2• 15 years ago

Flashblock isn't enough. The Firefox implementation lets the flash load, then hides it immediately.

maukdaddy2• 15 years ago

There is nothing more evil in modern business than marketers. Between real life experience and MBA classes I have come to despise most everything involved in modern marketing, especially in the technology space.

Alex3917• 15 years ago

Except for that virtually every single one of the biggest problems we face is a marketing problem, rather than an issue where we don't have the science or technology:

global warming, education reform, prison reform, the national debt, healthcare, literacy, food production, biodiversity, etc.

Marketing is probably the single most important career there is right now, and if there's any hope of humanity making it through the next thousand years then it'll almost certainly be due to improvements in our ability to market things rather than new technology.

ewanmcteagle• 15 years ago

For those examples are you saying that science or research is clear? I think it isn't for any of them.

jberryman• 15 years ago

> Except for that virtually every single one of the biggest problems we face is a marketing problem, rather than an issue where we don't have the science or technology.

Interesting point, and I agree.

> if there's any hope of humanity making it through the next thousand years then it'll almost certainly be due to improvements in our ability to market things rather than new technology.

If by our ability you mean progressives/the left, then I agree. Surely marketing per se is at best a neutral force.

dredmorbius3• 15 years ago

You're making the leap from "every big problem we face is a marketing problem" to "and the solution is to track users in minute and excruciating detail".

I can accept that there are ideas which are critical to the survival of the human race and/or modern civilization, which require mass education, and utterly reject your conclusion.

maukdaddy2• 15 years ago

But here's the problem - on a whole, no one is paying marketers to solve these problems. Marketers make their money by shilling for products that are the opposite of the issues you listed above.

For example, a newly minted MBA in Marketing needs to pay back school loans. Who has the money to pay them? Who has the job that will bring the most prestige and future earnings? The local farm selling grass-fed beef and organic products or P&G selling highly processed foods that are destroying the environment?

The marketer is going to take the P&G job and then spend their time and brainpower convincing people to buy even more processed crap.

teej• 15 years ago

Give me a break.

This isn't evil marketers hiding in their underground lair. This is web developers, designers, and product managers gaining insight about their users. They don't package this information and sell it wholesale to advertisers. They use it to make the product better.

You are taking this -way- too seriously. The ability to have perfect information on how users interact with your product is one the earth-shattering advantages we have as makers in the digital world. It means we can make something people want - better and faster than ever before.

I'm not saying that all use of tracking is good. All power can be used for good and for evil. But that doesn't mean that power is implictly evil. Save your rage for when you discover someone actually being evil.

jefft2• 15 years ago

Let's be clear: this ability isn't used to have perfect information on how users interact with your product.

It's used to allow publishers to make more money from their consumers through advertising.

goombastic• 15 years ago

I really miss not being able to down-vote you.

itg• 15 years ago

Sure there is nothing wrong with that, but these are sneaky underhanded tactics. Users should be able to opt-out if they don't want to be tracked.

cowboyhero• 15 years ago

> They don't package this information and sell it wholesale to advertisers.

Yet. Genie, meet bottle.

Thinking this information will never be sold is short sighted and naive.

When it becomes advantageous for them to do so, they'll sell it in a heartbeat and then sell it again. Somewhere in there the FBI and the NSA will start making "requests" for them to share with the government.

What begins as clever social commentary in movies like Minority Report tends to wind up as a sad fact of life years later.

a3_nm• 15 years ago

> Save your rage for when you discover someone actually being evil.

So it's okay for them to be evil as long as we don't discover that they are? More seriously, once they have the info, how can we be sure we nail them? The best way is still to ensure that they don't get the information in the first place.

thehotdon• 15 years ago

Marketers do nothing to "make the product better".

They pump exorbitant amounts of money into advertising, thereby increasing the final price paid by customers without increasing product quality. Furthermore, they use these advertisements to inflate their customers' perceived value of the products, once again, without actually increasing product quality.

Marketers maximize profits, not quality.

zedshaw• 15 years ago

Bullshit. If it was used to improve a product, they wouldn't use a tracking method that tracks me between sites. They can track what I do on their site, but if I'm going from goat-sucking-maggot.com to hulu.com, then Hulu has no business knowing that and knowing that doesn't improve their damn product.

This is entirely just an attempt to get competitive analysis about their competitors at the expense of user privacy.

RexRollman• 15 years ago

I usually use Firefox with it set to forget everything on exit, along with the Noscript plugin. Does anyone know if this tracking service would work on a FF user running Noscript?

By the way, using Noscript has made me aware of something that I didn't previously know: many sites call Javascript from lots of other domains. I've seen websites with as many as 18 other domains listed on the Noscript pull down menu. And I have seen an increasing number of XSS alerts as well.

sp3322• 15 years ago

All the social sharing buttons use 3rd-party JS. You can see the "embed" code for Google's new +1 button here: http://googlewebmastercentral.blogspot.com/2011/07/1-button-... This lets the sites update APIs without breaking every page on the Internet :)

NoPiece• 15 years ago

plus many sites use google's library api to load things like jquery! Used for good, 3rd party JS is a helpful thing.

http://code.google.com/apis/libraries/devguide.html

ktsmith• 15 years ago

And disabling all those crappy social sharing buttons makes the pages so much better. From faster loading to less clutter there are a bunch of benefits. Sometimes I'm flabbergasted how crappy sites are when I sit down to a machine without a javascript blocker.

dredmorbius3• 15 years ago

Amen to JS from third-party domains.

I see this as biting us in the butt sometime. Maybe not today, maybe not tomorrow, but soon, and for the rest of your life.

What's more annoying is playing the "NoScript allow roulette" game of trying to figure out which domains/scripts you have to allow for some site feature to work.

sixtofour6• 15 years ago

This is why I don't bother with cookie monitoring at all, and why I plug my ears and say "la la la" and pretend that everything will be alright. I really don't want to spend the time figuring out how to make my bank work.

I suppose when the government gets in the game, either through direct tracking or just making laws requiring tracking companies to keep particular data for particular lengths of time, then it will be a civil liberties issue and I'll care more. But it will probably also be illegal by then to circumvent tracking.

But that's just crazy. That would be like the government demanding that ISPs keep credit card information on their customers.

justincormack5• 15 years ago

You might want to remove all caching. Though deleting cache on exit will disrupt tracking if you exit often.

The etag mechanism will return each user a different etag for a piece of content, so the browser will send an etag changed request with that etag in. This will be stored with the browser cache.

Hisoka• 15 years ago

Doesn't this achieve the exact same purpose as logging a combination of the user's IP address + user-agent + maybe some other stuff? Don't need no complicated, cunning technology to do this...

______• 15 years ago

Exactly. On http://panopticlick.eff.org/ you can see how 'unique' your browser configuration, ip address, language settings, etc are. For most people, this creates a great many bits of information that can be used to track you even without cookies or any client-side storage.

SoftwareMaven• 15 years ago

Except this can do it cross browser and when I move from hotspot to hotspot, too.

code_duck2• 15 years ago

Why single out Kiss Metrics? One example, I visited Fox News last month and found they set up an HTML5 database called, in a rather unsubtle choice, "evercookie". I can't confirm that this is the case currently, though since the ability to view HTML5 databases in Preferences seems to now be missing from all the browsers I have (which seems odd, too!).

rhubarbquid• 15 years ago

In Chrome you can bring up the Developer Tools (View menu on the mac, I think it's in the wrench menu on other platforms). You can see databases, cookies, etc. in the Resources tab.

dredmorbius3• 15 years ago

"Evercookie" is more than just standard browser cookies: http://en.wikipedia.org/wiki/Evercookie

code_duck2• 15 years ago

Thanks, I see that in the Inspector now.

I recall menus in Safari, Mobile Safari, Firefox and Chrome which listed all the databases stored, along with the name. It was in Preferences near the cookie and password management.

It looks like the 'databases' menu is no longer in Mobile Safari preferences, and now Safari 5.1 will tell you what a website is storing in general terms, but no longer details the individual databases in preferences.

mambodog• 15 years ago

I imagine it is named as such because they are using evercookie[1], off the shelf.

[1] http://samy.pl/evercookie/

pavpanchekha• 15 years ago

I tried to do my best figuring out what this cunning new method is, but the article seems to have no information. Is it just that it's using my browser's ETags cache?

Also, what's with referring to ETags as a "theoretical technique never before seen in the wild"? It's pretty friggin standard.

rjbond3rd2• 15 years ago

The trick is the server generates a unique Etag for each visitor.

Then the visitor's browser sends the Etag back to the server (in an "If-None-Match" header), and thus it acts as a quasi-cookie.

Groxx• 15 years ago

That's the picture I get too, but I don't see how clearing the cache doesn't, you know, clear the cache. It would seem to imply that if the Etag is still around, it's not really cleared - maybe the data is gone, but the knowledge that the data existed isn't. And it persists through privacy-mode.

Which means I/we am/are either misunderstanding something, or the people who designed privacy and cache-clearing tools had a massive blind-spot.

sp3322• 15 years ago

KISSmetrics has a post explaining how the tracking works. http://www.kissmetrics.com/how-it-works They claim that simply using AdBlock is enough to defeat the tracking. They also claim "KISSmetrics has never, and will never, share anonymous customer activity of what people did on customer A’s site with customer B."

CrazedGeek• 15 years ago

One important detail for AdBlock: you HAVE to be using a Tracking/Privacy filter subscription. Please, if you're using ABP, add one of these as a subscription: http://www.fanboy.co.nz/fanboy-tracking.txt (Fanboy's Tracking List) or https://easylist-downloads.adblockplus.org/easyprivacy.txt (EasyPrivacy). None of the default filter subscriptions block KISSmetrics, but either of these will.

angryasian• 15 years ago

well the wired article directly contradicts what they say they are doing

"These services are using practically every known method to circumvent user attempts to protect their privacy (Cookies, Flash Cookies, HTML5, CSS, Cache Cookies/Etags…)"

They may not share information about specific users, but doesn't mean they don't use it to sell information in some aggregate form.

redthrowaway• 15 years ago

>They claim that simply using AdBlock is enough to defeat the tracking

I'm highly suspicious of that claim. The only site I have whitelisted is reddit, and I found the i.kissmetrics.com cookie in with the rest. That's not to say reddit isn't using them, but I'd be surprised given their very cautious approach to advertising.

fungi• 15 years ago

hazza for adblock, as much as i like to see the legislature and courts support privacy this is fundamentally a technological problem.

mooism23• 15 years ago

I disagree. This is a Whack-A-Mole game, and that means a societal solution is required.

Compare with spam. Technological solutions have reduced the problem, but to virtually eliminate it requires global law enforcement.

coffeedrinker• 15 years ago

Ghostery blocks many tracking scripts.

http://www.ghostery.com/download

newman3142• 15 years ago

Unfortunately, this does not help with mobile devices which do not have the same capability.

Well, I guess you can always maintain a hosts file but that's the only way I can think of.

aerique2• 15 years ago

There's Firewall iP for jailbroken iPhones. Also one can install other browsers than Safari from the Appstore and, for example, iCab Mobile has the ability to use filters (and comes with some).

bandushrew4• 15 years ago

" They also claim "KISSmetrics has never, and will never, share anonymous customer activity of what people did on customer A’s site with customer B.""

but if they change their mind, there is no way I can stop them - right?

watchandwait• 15 years ago

And there is nothing from stopping other companies from using the same techniques to sell individual browsing data.

jvdongen• 15 years ago

If, as the article implies, the same user gets the same identifier across all KISSmetrics customers they ARE (implicitly) sharing customer activity between customers.

As the article states, customer A and B could simple put the information they each have about identity XYZ together. Without KISSmetrics having to be involved any further than in providing and maintaining the unique identity to each customer separately.

Even if it was not the intent of KISSmetrics for this to be possible (which I've a hard time believing), the chosen implementation makes it possible.

meatsock2• 15 years ago

"I would be having lawyers talk to you if we were doing anything malicious." -- this seems like the type of defense that a good lawyer would tell you never to use.

bhrgunatha• 15 years ago

> So if a user came to Hulu.com from an ad on Facebook, and then later, using a different browser on the same computer, visited Hulu.com from Google, and then at some point signed up for the premium service, KISSmetrics would be able to tell Hulu all about that user’s path to purchase (without knowing who that person was).

It seems their method relies on using cached javascript files to identify a user. How then are they able to track the same user using a different browser? Is it by IP address?

zerd• 15 years ago

They can use browser fingerprinting, for instance. http://panopticlick.eff.org/

inportb3• 15 years ago

What happens when you login using both browsers? Now, Hulu.com can attribute both Km UID's to your account. Magic.

robtoo2• 15 years ago

How then are they able to track the same user using a different browser?

Flash cookies. Presumably Silverlight has an equivalent.

(And I even heard once that Windows Media Player shares cookies with IE regardless of the browser that it is embedded in.)

jscheel• 15 years ago

Jeez guys, not all tracking is evil. You know all that awesome content that exists on the web? Well the people that make and distribute that content need information to make your experience better. Let's say you start a new site. Let's use 8tracks for example: they provide a two-tiered service, one free and premium. The free service exists to drive you to a paid account, but you still derive value from it, nonetheless. In exchange for that free value, you give them stats that they use with their advertisers, who in turn give them cash they can then use to make your experience better. It's a give and take system. Thankfully, money isn't the only currency on the web, a little bit of info and some advertising goes a long way. I am willing to trade value for value, it's fair that way.

scythe2• 15 years ago

>not all tracking is evil.

Tracking isn't evil. Tracking people who specifically do not want to be tracked is evil.

inportb3• 15 years ago

You mean, people who take unfair^ advantage of freemium services?

  ^ according to jscheel's assessment of fair trade

nikcub2• 15 years ago

that is true. they knew that some users block all third party cookies and they still wanted to track them, hence using Etag

klbarry2• 15 years ago

What if I specifically don't want you on my website if you won't let me track you? You're using technology to circumvent me (adblock), why can't I use technology to circumvent your wishes (evercookie et. al.)?

bandushrew4• 15 years ago

that is very cool. The important thing about the system is that it is a matter of personal choice, right? I can choose not to participate, right?

because they ask my permission, dont they? I mean, I can choose between not seeing their contact or being tracked - yes?

inportb3• 15 years ago

Absolutely. You can easily choose not to participate.

If, on the other hand, you choose to deny access to your data while continuing to use the Web services in question, you would be at fault (using grandparent's definition of fair trade).

meatsock2• 15 years ago

"You know all that awesome content that exists on the web? Well the people that make and distribute that content need information to make your experience better."

that's funny, most of the web content i use most heavily was 'made' and 'distributed' without any input from the marketers / advertisers / trackers whom you are defending. not all tracking is evil, but the evil tracking offsets the good stuff by a wide margin. not all gun owners are evil but if you're running a company that uses that defense to justify business practices that require an invisible gun owner to be in my living room, i'm not going to be caring about how much my living room experience has improved.

slowpoke• 15 years ago

  Well the people that make and distribute that content need information to make your experience better.

If somebody wants my feedback, they can ask for it, and if I have the time and like/value their service, I will gladly comply. Simply 'taking' my feedback doesn't sit well with me.

sixtofour6• 15 years ago

There was a TV executive a few years ago who stated that getting up to go to the bathroom during a commercial is theft.

It's the commercials that make non-premium TV possible in its current free-ish structure. It seems that broadcasters should run their business with the assumption that a certain number of people are going to go the the bathroom and miss some commercials. And web companies should run their businesses and still be able to provide their free-ish services if some people decide to opt out of tracking.

What if everyone went to the bathroom all the time and missed all the commercials? That doesn't seem to happen.

underwater• 15 years ago

Looks like it's using a variant of a technique I demonstrated a while back: http://joshduck.com/blog/2010/01/29/abusing-the-cache-tracki...

thezilch4• 15 years ago

I'm not sure these researchers understand how private-browsing functions. The session in a private-browsing window is only private from the non-private sessions and only private from future private-sessions when all private sessions -- private-browsing windows -- are destroyed.

http://imgur.com/a/LjjYf

Here I have a non-private session, where I have request i.js (a second time), invoking an If-None-Match check with my non-private ETag of i.js. Opening a private session, my request to i.js does not invoke my non-private session's ETag and subsequent If-None-Match -- i.js is fetched as if my session has no memory of the URI.

In the second shot, I had closed my private session opened in the first test, and I then opened a new private session, without closing my previous non-private session. Again, my private session requests a new i.js, with no idea of the non-private session's nor the first, now closed, private session's version.

The onus is on browsers to restrict inner-private-session storage from leaking between tabs, but it could be quite messy.

mooism23• 15 years ago

Does it work that way in other browsers too?

thezilch4• 15 years ago

FF5 and IE9 function similar. Non-private and private sessions will not cooperate on the same cache, cookies, ETags, etc. Closing a private session will destroy all local cache, cookies, ETags, etc and is not reinstated when starting future private sessions.

joeshaw• 15 years ago

The main exceptions to this are Flash cookies. These are shared between all browsers for a given user, since they're stored by the Flash plugin itself and independent of individual browsers' profile storage.

joshtynjala• 15 years ago

"Starting with Flash Player 10.1, Flash Player actively supports the browser's private browsing mode, managing data in local storage so that it is consistent with private browsing. So when a private browsing session ends, Flash Player will automatically clear any corresponding data in local storage."

Source: http://www.adobe.com/devnet/flashplayer/articles/privacy_mod...

Local storage here refers to "Flash cookies".

k33n• 15 years ago

I've worked with the KM folks. Great people, genuinely kind, and they want to make a great product. I think it's disgusting to single out a startup like this, especially right as they are gaining traction with some big-name clients.

There is value in what they are doing, and there's absolutely nothing wrong with it. They are tracking user behavior completely anonymously.

AndyIngram• 15 years ago

If a user requests not to be tracked they should not be tracked. Even when the information is harmless, as I am sure it most likely is in this case, it sets a bad example and will make it worse for the industry.

muppetman• 15 years ago

Yea, they are _now_. How quickly that can change.

kevinchen• 15 years ago

Of course there's value in what they're doing — to advertisers. As a user, I am not 100% comfortable with any tracking service, supercookie-based or otherwise.

[Edit: Made the wording clearer]

rubeng• 15 years ago

KISSmetrics specializes in funnel analytics, not helping advertisers. I'm surprised by the number of people making that assumption.

I have a SaaS app and I use KISSmetrics to learn what sorts of things engage visitors and customers the most. It's helped me make critical decisions that benefit both me and my customers (by improving multi-step processes).

anyidiot2• 15 years ago

They are tracking user behavior completely anonymously

Just because you, a human, cant look at the millions of data points and go "oh look, there's george tomlinson of 28 esperay avenue doing something we dont like" doesnt mean that it cannot be done or will not be done, or indeed is not being done already.

Some of us dont want to walk around with yellow badges thank you. Do you imagine that fact that the badges are only visible to those with the resources and motive to discover them, and not the average joe, is more, or less of a motivation for privacy?

bandushrew4• 15 years ago

There absolutely is something wrong with what they are doing. They are deliberately going out of their way to stop me from enforcing my personal privacy requirements on my browser running on my computer over my network.

If I installed an application on your computer that sent me the names of all the applications you ran, when you ran them and how long for. That reinstalled itself when you attempted to remove it and that used every technique it could to gather information about you, you would want me arrested as a hacker.

I think what they do is disgusting, and I hope to hell that stories like this kill the traction they have gained with some big-name clients.

potatolicious• 15 years ago

Road to hell is paved with good intentions, etc etc.

I'm not going to make any moral judgments about their team since I don't know any of them. Even if they are on the up and up and mean the best, IMO the technology they are employing is illegitimate at best - it violates user expectations when using the internet, and IMO makes the industry more dangerous for the rest of us by giving legislators and luddites more ammunition. I am absolutely against supercookies.

I'm not sure why it's relevant that they're tracking users anonymously - the user gets to decide at the end of the day who they share their information with. To make this decision for them is presumptuous, to force them to comply despite their implied non-consent is the height of arrogance.

joe_the_user• 15 years ago

I'll leave discussion about valid of KM's activity aside, what is it about the "I know X, genuinely great people and really want to do good stuff" formula (with no substantive arguments) that seems to make to the top of these hn debates? (I'm sure the KM people love puppy dogs, etc).

reinhardt2• 15 years ago

Indeed, it's annoying. I'm sure most spammers, phishers etc love their mom and are cool guys when you meet them over coffee or drinks. It says nothing about their business ethics.

tomjen3• 15 years ago

If there is nothing wrong with what they are doing, why do they force it on me?

Actually, given what you said in that post, why do you force it on me?

Don't bother deny it -- you wouldn't know if they are getting traction if you weren't working with them.

techiferous• 15 years ago

"This is yet another example of the continued arms-race that consumers are engaged in when trying to protect their privacy online..."

I don't think arms race is a good analogy here. Arms race is a good analogy for virus-makers and antivirus software, since their goals are exact opposites.

The goal of analytics sites like KISSmetrics is to measure and understand the behavior of their customers as a group, not as specific individuals. The goal of people who wish to remain untracked is to avoid having personally identifiable information about them stored without their consent. These goals are not opposites and don't necessarily result in an arms race.

anyidiot2• 15 years ago

You know, when individuals access a company's computer using technically valid means (e.g. a username and password or by logging in from multiple locations), then its criminal charges, international arrest warrants, and jail time. [1] [2]

But when companies do it to people, oh its just a clever programming trick, and its not a problem because you could install additional software to prevent it from happening [3].

The law is showing up pretty clear that simply because you can access a computer system, does not mean that you may, and indeed that doing so without the user's permission is a crime. Causing a computer to store data on a user and then serve that data back to another computer seems dodgy without permission. Doing it when the user has taken reasonable steps to prevent it from happening? Class action time!

[1] http://www.techdirt.com/articles/20110722/02351315202/how-ci...

[2] http://www.geek.com/articles/geek-pick/aaron-swartz-spent-mo...

[3] http://www.kissmetrics.com/how-it-works

losvedir• 15 years ago

I generally browse with Javascript, cookies, and plug-ins off (except for a few whitelisted sites). From what I understand of the technology (it loads some javascripts initially), I think that would dodge it.

driverdan• 15 years ago

What's it like living in 1994?

sixtofour6• 15 years ago

The counter might be, what's it like living in 1984?

jacques_chester2• 15 years ago

Sorry, but no. You'll get tracked through ETags at least; if you have a late-model browser you may also be tracked through an HTML5 DB or history object.

fractalcat• 15 years ago

Title is misleading. I routinely 'dodge' this - all it takes is disabling caching. If you understand how caching works, it's trivial to conclude that it's possible to use etags for tracking. It's the same with the CSS-based browser history attack - if your browser is storing data, and it's possible for a server to tell you're storing it, it can be used to track you.

vl• 15 years ago

Ironically, they would never be caught if only they assigned different blobs to the same user on different properties, like KS_cookie XOR hash(property_name).

braindead_in• 15 years ago

What's the cunning part? I skimmed the article and it seemed to have everything other then the technique.

ZoFreX• 15 years ago

It's talking about two separate issues which initially confused me, one of which is inappropriate data sharing.

The cunning part technically is their repurposing of "etags". These aren't that widely known about but it's a mechanism by which you can ask a webserver "I've already downloaded this file before, has it changed?". Typically the etag will be a revision number, or a hash of the file. The header to create one looks like this:

ETag: "686897696a7c876b7e"

And then in future requests your browser will include the header:

If-None-Match: "686897696a7c876b7e"

In the request. If the file hasn't changed since you last downloaded it, you get a 304 Not Modified. Given that you can store absolutely arbitrary data in the ETag, it's easy to see how this can be used to track users (and the same applies to the Last-Modified header, which is treated exactly like an ETag by your browser despite containing a date).

meow• 15 years ago

Whats next.. tracking users using browser exploits ?

alexwestholm• 15 years ago

Sorry but what a bunch of crap... Privacy people are so annoying... If your concerned about this kind of tracking stop using online porn - otherwise As you were

nikcub2• 15 years ago

This has been known about for years, and was a concern on various mailing lists years ago. The solution at the time was said to be that browser vendors will build in tools for cache control in the same way they have for cookie controls.

The first sites to exploit this were, as always, porn sites. They used Etags in referral tracking to avoid webmaster fraud. (the webmaster would have to include a script from the affiliate co which would set an Etag).

You know what is more interesting? The Last-Modified header. The HTTP spec says that you are supposed to put a date in there, but it also says not to bother parsing the date if you are a client since date parsing is such a pain in the ass. So clients just copy the date string and store it and then replay it subsequent requests.

you can put whatever the hell you want in a last-modified field and all browsers will just store it and then replay it later in subsequent requests to the same resource. for eg.

initial request:

  GET /_modified_test HTTP/1.1
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
  Accept-Encoding: gzip,deflate,sdch
  Accept-Language: en-US,en;q=0.8
  Cache-Control: max-age=0
  Connection: keep-alive
  Host: localhost:8888
  User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_6) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.830.0 Safari/535.1

initial server response from my dev server (note Last-Modified header used):

  HTTP/1.0 200 OK
  Server: Dev/1.0
  Date: Sat, 30 Jul 2011 11:48:25 GMT
  content-type: text/html; charset=utf8
  Last-Modified: random_token_i_set
  Cache-Control: no-cache
  Expires: Fri, 01 Jan 1990 00:00:00 GMT
  Content-Length: 1634

subsequent browser request to the same resource:

  GET /_modified_test HTTP/1.1
  Host: localhost:8888
  Connection: keep-alive
  Cache-Control: max-age=0
  User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_6) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/14.0.830.0 Safari/535.1
  Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
  Accept-Encoding: gzip,deflate,sdch
  Accept-Language: en-US,en;q=0.8
  Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
  If-Modified-Since: random_token_i_set

with new webapps now being single-page with either hashchange or pushstate support, it means almost all requests are made on the backend to the same resource, so you can track the user across all pages on the entire site and across other sites.

concerning, but a known problem. even with these headers patched there is still a lot of information that can be used to fingerprint clients (ie. having everything switched off is still a fingerprint that makes you unique). I don't think chrome, safari, IE or Firefox will ever implement these advanced features, it will be up to somebody else to release a browser that is more privacy aware or to maintain a plugin that is.

I wrote a plugin that does this, but a lot of information still leaks through (it is in my github but I haven't released/announced it in any way). I am contemplating just forking webkit and doing a whole separate 'privacy aware' browser but haven't found the time. in short, the browser makers know about this, and have known about it for years - there is just no real interest in providing user tools to fully anonymize users.

Edit: if anybody is interested in the plugin it is here: https://github.com/nikcub/Parley

it blocks all third party requests and provides other features. it works, just needs a bit of a clean up and release.

speleding• 15 years ago

Interesting. I looked at the RFC and it says it has to be in the format "Sun, 06 Nov 1994 08:49:37 GMT" (RFC2616/RFC1123).

But even if browsers makers would check the "If-modified-since" against that format then it would still be doable to give each visitor a slightly different date and track them that way.

Combining the date stamps on 2 or 3 jpg or css files present on every page on the site should give you enough entropy for even the highest traffic sites and make it very hard to detect.

Cherian_Abraham• 15 years ago

Analytics is here to stay. Unless this practice is regulated (which in turn can end up being heavy handed and far reaching and in turn could discourage innovation) analytics will remain a big piece of what IT will focus on, mainly in getting a 360 degree view of their customers.

Instead of regulating everytime we see a practice that we may not agree on, how about we treat it like when the "iPhone location" fiasco broke. Do not criminalize the possession of customer data or even tracking, criminalize distribution or malicious use of it. If Company A wants to know where I came from, so that they can share their ad dollars effectively, I am ok with it. But do ensure that they dont share it with other companies in that network (whether Kissmetrics or someone else) for any reason. My online identity remains my own, it does not need to be dissected for further analysis by doubleclick, kissmetrics et al.

mattmanser2• 15 years ago

I disagree completely.

I bought this computer. I pay for my internet connection. And someone like KISSMetrics wants to spy on me using MY stuff?

To profit from MY computer tracking me against my express commands? Incognito mode, cookies turned off and they're tricking my computer into tracking me?

These are people who have lost all perspective of what's right and wrong.

Analytics is a solved problem, there's no innovation here, there's cookies and a way of opting out of it. If regulation is what's needed to stop scum like Kissmetrics from violating my privacy, then regulation's what's needed.

reinhardt2• 15 years ago

I bought this computer. I pay for my internet connection. And someone like KISSMetrics wants to spy on me using MY stuff?

You may pay for your computer and internet connection but not for the (vast majority of) sites you visit. This popular sense of entitlement is problematic when "your" stuff live in 3rd party servers running 3rd party software that you're not paying for.

aj700• 15 years ago

Peerblock can be set to block port 80 by all list or leave it open. I want to be able to enable some blocklists for 80 but not others. So I can block ads and stuff like this at the stack instead of the browser, but leave the other lists affecting only other ports, for torrents etc. I don't think it makes peerblock too complex to have some lists that block everything and some everything but 80.

slowcpu• 15 years ago

"Then, if that user eventually signs up during a later visit, KISSmetrics will associate their previously anonymous profile with their email address or user name. Which means that site admins can look at both how a user is currently using their site, and how they used it months or years before they actually created an account"

46Bit• 15 years ago

The issue is clearly not that they're tracking. The issue is that they're going to extremely devious lengths to prevent you from removing their ability to track you using standard tools.

I've quite a few /etc/hosts entries, blocking third party cookies, clearing cookies & cache on close, no flash cookies, and so on, but I always expect they'll be something they can find still.

danielharan• 15 years ago

Wow, an effing moral panic here. I thought KissMetrics was a darling startup?

Anyways, assuming they could offer their service tracking only on a customer's site, they should be serving from a subdomain, no?

ashkan2• 15 years ago

great comments.

we're planning to follow up with a post that has the technical details of the Etag stuff (sorry about 'light on detail', it was a press piece after all).

you're right in that it's been a known method that has been written before (samy had it in evercookie which we site in the paper and a few others have blogged about it). what seemed new (at least to me) was actually encountering it 'in the wild' on a top50 site like hulu. if this type of thing been written about before, definitely let me know so we can cite it.

fwiw, yes noscript would block the javascript that kissmetrics uses to respawn using html5/etags, however there's still the swf that regenerates using flash cookies. also josh highlights ways the you could do this with javascript disabled using CSS (kissmetrics actually also uses hidden values in CSS as well if you look at the src)

either way, blocking javascript/flash would render hulu, and other 'rich media' services like it, largely useless unfortunately.

RE: foxnews/polldaddy. actually they were naming their database 'evercookie' some time ago although they've seemed to have changed that (now it's just called pd_poll__). you can see the script they use here which they use html5 and swf databases: http://pastebin.com/0ieZ2i22 (prettyfied from http://static.polldaddy.com/p/4424060.js )

it's likely that polldaddy/foxnews are using these techniques so to ensure that a given computer only gets to vote 'once'. however, i think there are probably much better ways to do this.

hope that helps. i'll link a blogpost down here somewhere (which means that i actually have to start blogging finally ;)

ashkan2• 15 years ago

I put together a detailed follow-up on the KISSmetrics/Hulu respawning mechanisms outlining exactly how they work (although this is probably pretty basic for most the audience here).

Details here: http://ashkansoltani.org/docs/respawn_redux.html

Feel free to send comments/suggestions.

Also nikcub - very enlightening about the Last-Modified header! It reinforces my point that the solution to all this might not be technical but require policy guidance as to best practices, etc.

Researchers Expose Cunning Online Tracking Service That Can’t Be Dodged

Related Stories