How Cambridge Analytica’s Facebook targeting model really worked

http://www.niemanlab.org/2018/03/this-is-how-cambridge-analyticas-facebook-targeting-model-really-worked-according-to-the-person-who-built-it/

by Dowwie • 8 years ago

427 193 8 years ago

niemanlab.org

scottybowl• 8 years ago

I'm excited for GDPR.. The hard part is going to be getting the truth out of these companies about the actual extent of the data they hold on us

pavlov• 8 years ago

The EU should do a unionwide ad campaign pointing out the fact that the much-maligned eurocrats have actually been working for years to fix this very difficult problem that's only now becoming apparent to the wider public. The timing couldn't be better as GDPR goes into effect just after the Facebook/Cambridge scandal.

Unfortunately all EU institutions are terrible at marketing. If they did an ad campaign, it would probably be a TV commercial showing Jean-Claude Juncker giving a speech with subtitles in 15 languages.

rossdavidh• 8 years ago

That "hard part" is going to be more than just hard; I think the word you're looking for there is "impossible". They don't have any way of knowing what data Google, Facebook, Amazon, or any other company has. As this article does a credible job of explaining, you have to understand a fair amount about statistics (PCA) and machine learning to even know if if you were looking at it, and they won't know where to look for it. They have no enforcement mechanism in mind, and they passed a law anyway, which effectively means "you can't admit to having this", which will mean that the more willing a company is to lie, the bigger an advantage they will have over their competitors.

rmateus3• 8 years ago

Sure. But at least when they are actually whistleblowed (by people who "just" work for the company) there is a law which can be used to call to a court the theople who are legally accountable for the company

daenz2• 8 years ago

Am I understanding this correctly? Facebook user data (likes/profile info) was scraped to produce low-dimension feature vectors for users (similar to word2vec). These feature vectors were then run through some ML model to predict...what exactly? Targetability for effective political ads?

CiPHPerCoder• 8 years ago

[ Deleted. Nothing I say on HN ever matters. Move along ]

daenz2• 8 years ago

"What sort of lies" is pretty hand-wavy when it comes to labeling training data for a model. Are you summarizing from a source? I'm interested in the technical details of what happened.

emodendroket13• 8 years ago

Very few "undecided" voters truly are; elections are won and lost by getting your supporters to go to the polls. So if you wanted to use scurrilous, fake news to help your candidate, you'd be better off sending stories that will get your supporters really fired up and eager to vote and get their friends to vote, not trying to persuade the practically nonexistent undecided demographic.

emodendroket13• 8 years ago

It seems like the purpose was narrowly tailoring messages, which is something political campaigns are really keen to do now (Obama's campaign was kind of a trailblazer here, right?).

jacquesm5• 8 years ago

Obama's campaign was kind of a trailblazer here, right?

It's a pretty big gap between using and abusing social media and as far as I know Obama's campaign did not 'narrowly tailor messages'. They did target broad groups using generic messages and they did quite effectively use social media presence to build support.

But they did not - as far as I know, so please correct me if I'm wrong - go so far as to single out individuals or really small groups with the express intent of flipping their votes or targeting them with disinformation in order to try to stop them from voting.

And Cambridge Analytica seems to have been doing just that if the currently available information is to be believed.

r00fus2• 8 years ago

No it was Hillary's campaign in 2008 that was big on "microtargeting".

ie, moving beyond "soccer moms" or "defense dads" but "soccer moms with one kid and expensive tastes".

alexbeloi• 8 years ago

They used it to predict political affiliation of people that don't explicitly state a party preference.

The two parties already have a list of registered party members (and they can see who on Facebook explicitly states their party preference), for those members the main goal is higher turnout (they are the training data). The other voters they're interested in are unregistered (e.g. independent) voters that are likely to be on their side ideologically.

The core idea is very simple, they believe that if someone says they're independent, but their preferences/features (age, gender, location, likes, posts) are predicting moderate or high likelihood of $PARTY affiliation, then showing this person political ads may move them from 'maybe vote for $PARTY' category and get them in the 'definitely vote for $PARTY' category.

If you have continuous access to new Facebook data as you're serving ads, you can verify your ads are working on an individual basis by checking the predicted 'score' for $PARTY affiliation predicted by your model before and after an ad (I want to stress that this can be done on an _individual basis_). The likely sequence of events is that they did AB testing on different kinds of ads and found that fake inflammatory ads were most effective at achieving this goal in a very measurable way ($PARTY score), the resulting media/political atmosphere is collateral damage (hopefully unintended).

Source: I am a data scientist / machine learning scientist and this is how I would do it and how it seems to me others. I don't work on political data but I have worked on personalized recommendations which are similar.

demirev• 8 years ago

Did the app really grant them continuous access to user info? I thought it was a one-off thing - they get your data at the time of use (and your friends') and that's it.

Plus the approach you outlined would require the user to like/dislike things based on the add they saw, so CA can observe a change in the predicted affiliation (they didn't have access to posts as far as I know). I don't think it would have that effect (even if the add influences you, I doubt that it would make you go unlike Obama's page for example). Not to mention that by any likelihood you shouldn't be able to verify that a particular add was shown to a given individual.

I suspect it was a simpler use case - they would group users into segments, and then craft different add strategies for each one (maybe based on other research or just expert opinion).

godelski8• 8 years ago

has revealed that his method worked much like the one Netflix uses to recommend movies.

I'm not sure this is the model you want to emulate. The suggestions are terrible and continually getting worse.

rjurney• 8 years ago

It's a hard problem but Netflix's model represents the state of the art in machine learning for recommendations. Still scared of the singularity? :)

nicoburns• 8 years ago

Really? Because in terms of actual usefulness, I find youtube's suggestions to better...

dazc5• 8 years ago

How hard can it be? Any film that I didn't watch all the way to end ought to be a strong signal that I didn't enjoy it. Not that I want to watch other movies just like it.

acdha• 8 years ago

There’s a bug confound with the catalog: Netflix’s streaming catalog is much smaller than their DVD selection was and it changes as licensing deals expire. No model can make up for that entirely.

godelski8• 8 years ago

Personally, I used to get substantially better results. It would suggest to me a movie I'd never seen before with a score above 90, I'd watch, and enjoy. Now I'm getting things like kids shows suggested to me. I double checked by history to make sure no one watched one on my account, but they keep popping up with high percentage. I also really like horror and get weird suggestions [1] for similar movies. After I watched Alien it suggested the Great British Baking show. I literally have 0 trust in the new ranking system. They push certain shows way too hard and TV shows I'm watching show up on page 3 of "continue watching".

This is far from my previous stance of "Huh, movie I've never seen before and in a different language? It's 95% so yeah, I'll give it a go."

And before someone says anything, I do vote up and down quite frequently. But can't help but notice that my suggestions were better when I had the more nuanced star system.

[1] https://i.imgur.com/MOt3XlL.png

[1.1] How I'd rate these. Cars 3: not really interested. Liked the first though. Stitches: idk, doesn't look appealing. Teeth: Classic cult film but yeah... Big Mouth: I have ZERO interest in watching this, please stop suggesting. I have downvoted this! Waterboy: I like it, but far from 95%. I'll give it like an 80.

pps432• 8 years ago

Yes, it's a hard problem, but no, Netflix recommendations are nowhere near state of the art. They were at the time of Netflix prize, but since then the field has advanced a lot, while Netflix only dumbed down their recommendation models.

cirgue• 8 years ago

There's a problem with all ML-based content recommendation, whether it's advertising, movies, whatever: the people who make the content have strong opinions about what should get shown to whom and when. If those people have negotiating leverage over your organization, your business team will compromise your recommendations to placate those people. That means that after you've build this beautiful model that minimizes whatever cost function you've chosen, there will be a bevy of business rules and content-specific score adjustments that will be overlaid. Over time, this shaggy, bad-assumption-laden system will dominate the user experience, and unless your management has a vested interest in maintaining the integrity of the recommendation system, your model will eventually be lost in a sea of human-generated noise. Not that I'm bitter about this or anything.

codezero2• 8 years ago

Are you saying all of Netflix suggestions are getting worse? How are you measuring that?

Or are you saying your netflix suggestions are getting worse?

Mine are pretty good and have held steady for a while, at least in terms of my own preferences, though I also am probably using it a little less as I've got Prime and Hulu now as well, so there's probably less times I'm randomly searching through Netflix and finding nothing.

godelski8• 8 years ago

I can only tell you what my personal experience is. I laid it out more in another comment under rjurney's reply.

I'll also add that I am more frequently searching for 15 minutes then switching to another service. I used to find a movie to watch in 5 minutes. I am a big movie person too, and will watch most things. But I am also more aware that if I watch one show that is just "meh", then I am going to be bombarded with shows of similar quality for the next few weeks.

Also, there is a fairly obvious pattern that shows up from the movies in "my list". Those do not seem to be weighted more heavily.

joncp• 8 years ago

It sure seems that way. It's almost as if Netflix is giving in to pressure from content owners (which now includes themselves) to downplay or even weaken their suggestions.

First, they got rid of those wonderful ranked lists that made us love Netflix in the first place, replacing them with the much more opaque cover art carousel view. Then they started mixing in lower-ranked items into the carousel. Finally, they switched from the five-star rating to the thumbs up and down buttons, which can't possibly give them as much information about your opinion.

godelski8• 8 years ago

Currently I have zero trust in their rating system. Actually, zero is a bad number because if I see a high ranking I think I am going to dislike the suggestion. This is opposed to me previously trusting the system a lot.

dmix3• 8 years ago

I've heard rumours that the internal backlash at Netflix against 5 star ratings happened with Amy Schumer's most recent comedy special which had thousands of 1 star reviews on Netflix. Which was one of their most expensive comedy productions and whose release coincided not long before the switch over to vague thumbs up/down.

Note, the comedy special was similarly panned across the press and social media as being repetitive of her previous work, extremely predictable punchlines, and when seemingly good later exposed to be highly derivative of other comedians work.

But regardless of the reality/honesty of her rating it was apparently not good for Netflix's business when they let users destroy content they produced destroy it on their own web site via user generated content.

So the suits (heavily swayes by their production studio and Hollywood) were able to convince the product team to hurt the UX for 90% in order to protect the popularity of the 10% of content they own.

The truth may be good for consumers in almost all situations. But sadly the interests of executives dealing closely with high value B2B partners and investors tend to have a way to out-valuing the interests of the average user (not to mention far out valuing power users).

So I guess we have to rely on 3rd party IMDb web extensions which inject into Netflix in order to get honest ratings.

vanilla_nut• 8 years ago

In my anecdotal experience, it seems like Netflix's own content gets weighed disproportionately high. This could be a result of the fact that first-party content never gets removed, so it builds up a more complex graph of recommendations, but personally it feels like they're trying way too hard to cram their content down my throat -- to a point of recommending shows that I would never even consider watching, like Jane the Virgin (not exactly my kind of humor). In the end it just feels like they've compromised their own recommendations to push their own product, which makes me mistrust all of their recommendations.

lordnacho2• 8 years ago

Finally, I was waiting for someone to talk about the model itself. It makes sense that SVD or something like it (PCA, co-occurrence, etc) would be used.

But I also wonder what exactly you are going to do with the predictions. What exactly do you show to someone to make them more likely to go and vote if they are inclined to vote your way, or make them stay at home otherwise? Is there evidence that whatever you're showing actually works? Or do you try to change people's minds? What do you do?

Knowing how the state of things -in this case, people's voting inclinations- is not the same as knowing what to do, ie a strategy.

I don't know how effective it is, I'd like to learn more. But I smell the possibility that these CA type firms are simply selling snakeoil to desperate political activists.

tomrod2• 8 years ago

What exactly do you show to someone to make them more likely to go and vote if they are inclined to vote your way, or make them stay at home otherwise?

Qualitatively: show things that get them angry.

Quantitatively: test and control pop splits.

mattmanser• 8 years ago

How do you test anything? There's only one vote, you can't iterate.

sebleon• 8 years ago

Quantitatively: test and control pop splits.

How do you actually do this? Presidential elections come once every 4 years.

NoodleIncident• 8 years ago

Shortly after the election, I read something saying that the actual ads were targeted soundbites at specific demographics likely to vote Democrat, run shortly before the election with the intention of suppressing voter turnout.

r00fus2• 8 years ago

So negative advertising aimed the core constituency of the opposition's voters speaking to their deep seated concerns about their candidate.

I could imagine this working on Dem voters who are wavering on Hillary with leads like "she thinks the TPP is the gold standard" etc.

RpFLCL• 8 years ago

One example I can provide is of gun control topics.

If you understand someone's mentality on the subject you can decide if they see:

1) An ad with someone breaking into a home and the homeowner defending themselves with a firearm (sell insurance?)

2) A grandfather and grandson on a hunting trip (hunting supplies?)

3) Or maybe gun violence hotline with powerful images.

The people seeing these ads are under the assumption that everyone else sees them, not that it's specifically targeted at their personality type. These affect if you think other people understand your issue or not. Thus affecting your motivation and attitude.

If you see an ad that fits your mindset, you think you're on the majority side. This was powerful in classic media, it's just as powerful now.

golemotron• 8 years ago

The people seeing these ads are under the assumption that everyone else sees them, not that it's specifically targeted at their personality type.

How long will that be true? Do people make that assumption about search results?

downandout• 8 years ago

I don't know how effective it is, I'd like to learn more. But I smell the possibility that these CA type firms are simply selling snakeoil to desperate political activists.

According to the article:

"The accuracy he claims suggests it works about as well as established voter-targeting methods based on demographics like race, age, and gender....the digital modeling Cambridge Analytica used was hardly the virtual crystal ball a few have claimed."

It's pretty clear that they were selling snakeoil. In fact, the use of CA wasn't particularly helpful to anyone [1]...hiring them was just a prerequisite for obtaining campaign contributions from the Mercer family, who had put up the money behind CA [2].

[1] http://www.businessinsider.com/cambridge-analytica-facebook-...

[2] https://twitter.com/kenvogel/status/975756418128187393

wu-ikkyu• 8 years ago

I don't know how effective it is, I'd like to learn more.

Here is an interesting Ted Talk which discusses an FB experiment that details how effective minor UI changes can be on voter turnout (13:40)

https://www.ted.com/talks/zeynep_tufekci_we_re_building_a_dy...

daodedickinson• 8 years ago

Door to door canvassers these days carry devices that tell you what topics to bring up and what topics not to bring up at a certain address, even distinguishing between individuals at an address; some are told to demand a husband let them talk to the wife, for example.

dpwm• 8 years ago

I don't know about the specific campaigns that you are referring to, but in my experience a lot of the information used in campaigns I've been involved in comes from previous canvassing sessions. Political parties in most countries are involved at many levels where there are elections. Canvassing doesn't just take place for the big elections.

One year they will have been round and had a lengthy discussion with Mrs X, but Mr X slammed the door in their face another time. This was somewhat lower tech: the information was printed out and attached to a clipboard.

Most of the time this information is correct. It's more interesting when it's really incorrect. That said, some of the best sessions I've been involved in were where there was no information.

fjsolwmv4• 8 years ago

demand a husband let them talk to the wife, for example.

Demand? That seems like a great way to get arrested or shot for trespassing.

sg0• 8 years ago

Before Netflix stopped showing the number of stars next to content, I used to sort of depend on it for choosing a movie. In fact, I sort of miss it now, and spend more time sifting through content undecided. That's because I am clueless about movies. I believe that there are people who are as unsure about electoral candidates (as in, at a given day, they don't favor one candidate above others) as I am about choosing movies. When push comes to shove (my wife's irritation quotient above threshold) in terms of making a decision, an advertisement that someone saw couple of days back can definitely assist in making a choice at the split second.

jpalomaki• 8 years ago

It is reasonable to assume that a marketing message written with the profile of the targeted person in mind works better than generic message.

In Facebook campaigns you can use certain things, such as user's interest, to select who sees your message.

I'm not an expert on Facebook analytics, but I believe you can get pretty good stats on how your campaigns are working, how much promoted posts get shared etc.

This sounds like the holy grail for advertising. You get to write your message for certain profile and get quick feedback how it worked. Even if the system is not perfect, you would have an advantage compared to somebody else who is spending the same amount of money and not using similar targeting.

Maybe their model also allowed them to find social influencers with many followers. Being able to targer these people and get them to share your message would be really good.

The article compares this to the effectiveness of traditional voter targeting methods. I'm not sure what the parameters used on those are, but maybe all of them are not available on FB, justifying the need for something else.

emodendroket13• 8 years ago

I am really puzzled by the Cambridge Analytica scandal. It's not particularly savory, but is there something happening here that it wasn't basically already known about how Facebook worked? By the protests of their own executive, the system was working as designed, and at worst Cambridge Analytica misled them about how they intended to use the data, right? There was no actual security breach here, as far as I can understand it.

jacquesm5• 8 years ago

There doesn't have to be a security breach for it to be a very bad example of using data collected in one way for a completely different purpose. It violates the 'lawful basis for processing' part of privacy legislation.

hueving2• 8 years ago

Which US legislation?

ikeyany3• 8 years ago

Who said anything about a security breach? Most of the controversy has been about the company influencing elections using data scraped from people (and their friends) unaware of what the data was being used for.

MattyRad• 8 years ago

The degree to which it influenced the election is questionable. Despite all the headlines, I haven't yet seen any convincing analysis of the impact of facebook on the election (I'm not sure how one would even go about doing so). So far it seems like it's just a convenient vehicle for people that dislike the outcome of the election to express indignation.

iagovar• 8 years ago

No. I know at least one political consultancy that does a similar work in Spain, although I only know they work with data, use micro targeting and is run by a sociologist.

frozenlettuce• 8 years ago

It became a "problem" because it helped Trump win.

vuln2• 8 years ago

This is exactly it. At least it stops the news from droning on and on about Russia.

I thought Clinton spent large amounts of money on data and the Democrats admitted the data was bad or at least that was their excuse. How much did CA pay for this data? I still find it crazy that Trump campaign spent 30% of what Hillary did and still won. The Russians used 100k$ worth of ads to sway the election. This stuff doesn't t add up.

tasty_freeze• 8 years ago

This answer ignores some known details of the story. Cambridge Analytica didn't just buy targeted ads from Facebook. They used a sockpuppet to release a fake "take a personality profile" app, which then allowed them to gather tons of data against the Facebook terms of use.

The CEO of Cambridge Analytica has also been recorded telling a (fake) potential client that they routinely blackmail people using prostitutes and who knows what else.

So unless you can show that Clinton's campaign was doing the same things, your claim is a false equivalence.

spitfire2• 8 years ago

When talk about CA first emerged on HN before the election some posters found the original papers referred to. They were looking at pictures in the story and zoomed in to find the titles.

I cannot find those posts for the life of me again. Not suggesting anything nefarious here, I just can't find them. Does anyone have a link to those early conversations or make copies of the papers?

I made copies earlier but deleted them before I put them into my papers archive.

erikpukinskis• 8 years ago

Here are some leads:

https://news.ycombinator.com/item?id=14486365

https://news.ycombinator.com/item?id=14393991

https://news.ycombinator.com/item?id=14330547

https://news.ycombinator.com/item?id=14284502

https://news.ycombinator.com/item?id=13939814

And the query: https://hn.algolia.com/?query=mercer&sort=byPopularity&prefi...

spitfire2• 8 years ago

No none of those are it. I believe it was before the elction that the article came out. There was a picture of someone reading the paper CA was supposedly based upon in the article. Someone zoomed in and found the paper.

I'll keep looking.

etiam4• 8 years ago

Spoiler warning. Article punchline ahead.

"The whole point of a dimension reduction model is to mathematically represent the data in simpler form. It’s as if Cambridge Analytica took a very high-resolution photograph, resized it to be smaller, and then deleted the original. The photo still exists — and as long as Cambridge Analytica’s models exist, the data effectively does too."

That's an eloquent piece of explanation of a very important point. And apropos the discussion about privacy legislation, it's also going to be a very interesting point. Will the Cambridge Analyticas of the world be able to claim they have held on to no personal data, when strictly speaking the raw data has indeed been deleted after being used to create a derivative work that can for all important purposes be used to recreate the original? Assuming I find out I'm being profiled and demand to have my data removed, will society grant me rights to have derivative forms removed or adjusted too? I'm somewhat pessimistic that legal hairsplitting about matters like these will make enforcement very difficult.

autokad• 8 years ago

"will society grant me rights to have derivative forms removed or adjusted too? "

I am in favor of no. Imagine I build a gender classification model off public tweets, and then you later delete your twitter account and demand my model not be used because it was trained off 'your data'.

I am in the camp that so long as the data isn't traceable back to you specifically, then don't put any information out there you are not OK with sticking around.

salty_biscuits3• 8 years ago

I guess the question is what are you trying to protect? The model is fundamentally lossy as it is a rank reduction method so your original data is gone (i.e. no one would be able to accuse you of liking a particular controversial post, just that you are likely to like that post). So it sort of has the differential privacy thing going on. I guess it is another question as to if such models should be built at all. I think the fidelity of the models will answer that in time, if they work really well it is scary, if they are poor models they will cease to be used. I suspect that it will be in the middle and highly sensitive to the quality of the original data and the quality of the implementation like all ml applications.

darawk12• 8 years ago

when strictly speaking the raw data has indeed been deleted after being used to create a derivative work that can for all important purposes be used to recreate the original?

To be precise, you almost certainly cannot use this data to recreate anything remotely resembling the original dataset. This type of dimensionality reduction would throw away enormous volumes of data. There is no meaningful sense in which you can reconstruct the data from it.

What they have done is distill some insights about people from this data. It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really.

It's honestly kind of disingenuous to describe dimensionality reduction in the way that they do here. It is like reducing the resolution of a photo, but it'd best be described as reducing that resolution to say, the 20 most representative pixels. There's no real sense in which the photo still exists.

anf4• 8 years ago

It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really.

So if Google has distilled someone's emails over the years into "closeted homosexual with a deeply repressed leather fetish", that's not an invasion of their privacy as long as they throw away the source materials?

berkay• 8 years ago

Analogy does not work here and is misleading. You cannot do much if anything with 20 most representative pixels (if there is such a thing) but you can infer highly valuable characteristics about the person. Yes, you cannot recreate the original data but what you end up is potentially much worse (sensitive/private) than the original data.

stochastic_monk6• 8 years ago

That's only accurate in the sense that because an LSTM's hidden layer is much smaller in dimension than the data on which it is trained, there is less information in it.

However, it concisely represents a manifold in a much larger dimensional space and effectively captures most of the information in it.

It may be (and is) lossy, but don't underestimate the expressive power of a deep neural network.

chii• 8 years ago

distill some insights about people from this data

i'd argue that insight is the bit that's important, and the bit that's the privacy risk.

stevenicr2• 8 years ago

" cannot use this data to recreate anything remotely resembling the original dataset."

This by itself may be mostly true perhaps - and many of the comments get into ways of playing with this dataset to make it better, I don't have experience with those methods, but,

what I have not seen anyone mention, if you have this dumbed down dataset, the original is gone.. you can still combine with other data sets that are either public or previously created and likely fine tune;

dumbed down set + public voter records + public arrest records + previous whatever records - sort, match, what's left over.

and pretty much recreate what you needed from the original, maybe not 100%, but I would guess you could get really close.

etiam4• 8 years ago

To be precise, you almost certainly cannot use this data to recreate anything remotely resembling the original dataset. This type of dimensionality reduction would throw away enormous volumes of data. There is no meaningful sense in which you can reconstruct the data from it.

First off, I think that's wrong. The idea is after all to keep the information that will result in the smallest error compared to the original on the dimensions one cares about. Within what the model emphasizes a reconstruction can be not only "remotely resembling the original dataset" but as closely resembling the original dataset as is possible with the capacity of the representation.

Next, I'm really not talking only about the particular method described in the post. It's definitely possible to choose to make a light enough reduction to preserve the aspects of the information one is interested in, and to optimize for recall rather than generalization. A more realistic context is going to be that some information about the affected individuals is still exposed or kept (maybe in a compact derived form), which would in many cases give excellent possibilities to restore information accurately enough that claims to have the removed the data are effectively deceptive.

Even for cases where the models are in good faith created only to "distill some insights" I'm skeptical that they really are useless for recovering individual information. I'm by no means an expert in differential privacy but I do listen when it comes up, and a lot of what we see from that field seems to come down to being able to trade off the relation between keeping the data useful and how many pieces of additional information (or assumptions and brute force) are needed to break the integrity protections. With surprises that tend to be on the side of 'Oops. Turns out this clever trick can recover the originals easier than we thought.'

It's honestly kind of disingenuous to describe dimensionality reduction in the way that they do here. It is like reducing the resolution of a photo, but it'd best be described as reducing that resolution to say, the 20 most representative pixels. There's no real sense in which the photo still exists.

In my honest opinion the original analogy does an excellent job of intuitively explaining that most of the informative aspects of the data are kept (we can still see just fine what's in the image) while irrelevant details are discarded, and that is probably what was intended.

If anything comes off as disingenuous in that context it's your representation that it's like a strong reduction in the pixel domain (where it does indeed destroy a lot of the information). What can be done is much more like running the picture through a high-performance Imagenet classifier and keeping the 20 (or 2048, or whatever's needed) most informative values at a level that corresponds strongly to semantic content of the picture, and holding on the model. We could probably generate images that people would have a hard time distinguishing from the original with that.

zaphos• 8 years ago

This type being ... something like PCA? It's up to the user how much to actually reduce the dimensionality.

The pixel analogy is bad, but to use it anyway -- you get to choose how many pixels you keep. You could keep literally all of them.

kurthr2• 8 years ago

So if I compress a BMP by 100:1 using "lossy" techniques then it's not equivalent to the original? I'd say that depends on how recognizable the result of reconstruction is, and not on the amount of reduction. MPAA would be very unhappy with your argument.

To be more extreme there are many compression/extraction methods that can perfectly reconstruct the original data with very high compression ratios. GIF/PNG can reproduce many images exactly. Certainly, they are derivative works?

pbalau• 8 years ago

If I tell you I like furry porn, is there a way for me to make sure you forget that? This has lots of implications, many of them placing the "blame" on me for telling you this.

flippy2• 8 years ago

That’s strangely specific....

Hmmm

etiam4• 8 years ago

I think that's a fair point, but I question if it's an apt comparison to the situation with some big players of the surveillance economy.

Let me make that a bit more convoluted for you:

Let's say, very hypothetically, that you, like most of the general population suck at things technological and just switch off mentally when someone mentions phrases like "social graph" or "Javascript", and I'm a semi-psychopath who plans to make a fortune by scamming dumb fucks like you. I take on my mask of sanity and most endearing nerd T-shirt and appear at your door to give you a FREE robot servant. Except to keep it FREE the robot servant is going to pause what it's doing sometimes and whisper subliminal messages to you from my sponsors. But you're not afraid of stuff like that are you? And all you have to do is sign my brick of legal documents in complexified Legalese, which of course you don't have mental stamina to read through. "But, hey", you think "people are nice and trustworthy and if there was something really bad going on here that would be illegal and punished, and besides what's the worst thing that could happen", and I get your signature and you got your FREE robot.

If you had bothered to learn complexified Legalese and do your reading you'd have noticed you also just approved that the robot spends it's spare computation cycles surreptitiously watching you and getting to know you, and one of the many things it does is it glances over your shoulder when you get your porn fix, and in collaboration with my team of highly trained robot masters it concludes not only that you're into furry porn but also which particular furries really push your buttons. We catalogue this away for future use. Years later, in spite of your technical ineptitude, but maybe let's say because you're really a good people person, you've risen to become the highly respected mayor of your town. Then a business partner of mine who used to work covert operations over at CI6 but now has switched to lucrative private contracting, comes to me asking for the files collected from your robot for a influence gig he's taken on from Toxico, and I sell them to him the data for a suitably juicy sum for a man of you stature. Ex-CI6-guy studies the file with interest and uses it to select two skilled furries he can tell you'll be incapable of resisting and sends them to cross your path at a representation dinner. They very convincingly persuade you to have them over the next weekend your wife's away, and it's WILD(!).

And of course "your" robot is carefully documenting the whole thing, which I also sell to my partner, for an additional cut of the profits from the Toxico job. While you still wrestle with your conscience about whether last weekend was really a Good Thing, someone appears at your office door to propose you use your influence to switch the city energy supply over to a 90-year contract on Toxico's patented owl-burning power plant (with levels of carcinogen emissions they'll never be able to get us for!). Incidentally that someone at your office door has probably also been chosen according to your robot file to be a kind of person you'd have a harder-than-usual time to say no to for one reason or the other, but in the end you still refuse because despite some personal weaknesses you're a decent man, who values and protects your town, it's people and it's environmental surroundings. And then you're informed that someone may have videos you'd really rather not become public. Unless you agree to the Toxico proposal and do so generously, your comfy little life may meet with a sudden and radical change of fortune.

So who's to "blame" in this scenario? I'm sure there's a point where ignorance should be illegal, but I think generally society looks with some leniency on getting deceived. And winding back to the early parts, what does this do for your moral rights to have me remove your data? It was right there on page 200 in crisp, clear letters that you allowed me to watch your porn surfing habits and share that data and derived works with selected partners. You agreed to this! We haven't done anything that we're not allowed to do according to the contract.

...

jobigoud• 8 years ago

This analogy immediately reminds me that with enough high res / low res pairs we can rebuild a high resolution image from its low res version with fairly good results. Wonder if the same could be done here.

mercer• 8 years ago

To take a completely different approach in terms of 'derivative works':

Say we have a bunch of profile images, and then describe them in text. "Blonde, caucasian, large nose, curls, receding hairline, strong jaw, big ears", or perhaps even more specific stuff like "has a mole on the left cheek at the same height as the right earlobe" and "right nostril is larger than the left" and "dimple in chink".

Based on a description like this, we could identify an individual in probably a short paragraph. Nonetheless, on the data side, this is a lot less information than is represented by the raw pixels.

When it comes to the topic of CA's tools, and 'psychosocial' targeting, we can't separate the broader context and the way in which one single term can encode tons of data ("looks like George Clooney with a bigger forehead"). I'd argue the same princple applies to political views, and personality.

kurthr2• 8 years ago

Based on that description I'd say it's closer to taking a RAW format and converting it into a JPG, which contains most of the (human visual) relevant data from a DCT while removing the remaining "noise"... which is a bit different from simply resizing (eg subsampling). What I would find a bit satisfying about this description is that you probably wouldn't get away with claiming that a JPG of an image was un-copyrighted and only a BMP (of the same resolution) was.

To be fair, it does depend on the amount of compression before it is not recognizable, but if you can still squint and see the Mona Lisa (when you also have her phone #)... have you not violated her privacy?

tunesmith• 8 years ago

Isn't the accuracy of the predictions kind of orthogonal to the fact that they were basically lying in their attempts to change behavior?

Using lies to convince someone to do something is going to be more effective than using truth, if that "something" is not in accordance with the truth.

The comparison with netflix really breaks down there. You're not going to be able to convince me that I liked Crash, so recommendations based off of that aren't going to be very useful to me.

But if you reinforce my false belief that Obama and Soros are gonna use the deep state to invoke Sharia Law on the 2nd amendment, then that might better convince me to vote for so and so.

willart4food• 8 years ago

archived for future reference http://archive.is/dMIcN

vcdimension• 8 years ago

This is the standard way of analysing this kind of data, and I'd be very surprised if the Obama campaign didn't use the same or very similar methods with the facebook data they obtained. The only difference is that Cambridge Analytica managed to obtain much more data over a wider demographic.

soared• 8 years ago

Very interesting article but I wish it went one step further. Why does it matter that cambridge analytica knew a user's big five or that they were an old, uneducated republican? How was this (inferred) data used?

I assume they wrote/created different ads for different sets of users... but how many segments did they have? Did their graphic designer build 500 different ads, or was text/images dynamically inserted based on these variables? How did they figure out which message would resonate with each segment? How did they test something like this, with so many potential variables? Was this knowledge used only on facebook, or across all digital channels? Was it implemented in non-digital channels as well?

I'd kill to have access to their campaign set ups.

cosmic_ape• 8 years ago

Besides the data, its interesting how the actual targeting was performed.

Does facebook provide an option to show a particular given ad to a particular given user? Or is it possible to select a group of people with a given set of likes? How fine-grained is facebook's audience selection mechanism for ads?

Or was the targeting performed by creating fake groups, befriending people?

albertTJames• 8 years ago

Yeah yeah, they only tried SVD on a multimillion dollar dataset. Psy-ops, netflix, same deal, what is all the fuss about! Strange that a research scientist is released of his NDA a couple of days after channel 4 documentary.

katebrooks• 8 years ago

And we can't blame the companies because we agreed our data to be monitored when we installed the apps and granted permissions. Facebook and Google has taken so much of our data and knows more about our behavior than we ourselves. Even if I demand my data to be removed I don't know if they will 'actually' delete it.

gleglegle• 8 years ago

Who says we can't blame them?

How Cambridge Analytica’s Facebook targeting model really worked

Related Stories