Meta was scraping sites for years while fighting the practice

https://www.bloomberg.com/news/articles/2023-02-02/meta-was-scraping-sites-for-years-while-fighting-the-practice

by mfiguiere • 3 years ago

337 116 3 years ago

jeoqn2• 3 years ago

…and? Of course everybody does whatever they can get away with.

radarsat1• 3 years ago

Interesting perspective. So in your view, no person or company with integrity exists? And no one lacking it should be called out?

aulin2• 3 years ago

integrity in capitalism only exists if some soverreign entity enforces or if it's the more profitable path (e.g. to avoid PR scandals)

jeoqn2• 3 years ago

We shouldn’t be surprised by lack of integrity. This is not news.

noizejoy• 3 years ago

no person or company with integrity exists?

… probably not in the most powerful x percent.

Dalewyn• 3 years ago

A person with integrity is either a saint or a liar.

A company with integrity has an amazing marketing department.

Karellen4• 3 years ago

It might be worth considering GPs comment as a case of projection, rather than a claim about the world to be rebutted. After all, they are part of the "everyone" they talk about. And the "everyone else is just as bad as I am, really" (also commonly phrased "we are not so different, you and I") is a classic asshole refrain.

When someone tells you who they are, believe them.

seydor• 3 years ago

Then we should all be able to get away with it right? Can't say there is no problem here

timeon• 3 years ago

We do not need to downplay the hypocrisy even if it was obvious.

tukantje• 3 years ago

...and? Your point?

Putting it to spotlight is an effort to make sure they can't get away with it. Even in a scenario where you say "of course they do it", jumping to the conclusion of "we should not report on it" is ridiculous.

JohnFen5• 3 years ago

Not nearly everyone. Plenty of people follow ethical codes.

creshal2• 3 years ago

Rules for thee, but not for me, as always. All big established companies want to ensure that nobody coming after them has a fair chance.

polotics• 3 years ago

Peter Thiel's infamous "competition is for losers" indeed comes to mind.

dbuder• 3 years ago

Be the biggest hypocrite possible seems to be his motto

themitigating3• 3 years ago

That's what businesses do. They are created to make money. They only way to keep them in check is with regulations. The unfortunate thing is so many people hold libertarian views while others are anti-big government.

m0llusk• 3 years ago

They may call themselves Libertarians but there is this competing idea that the path to liberty necessarily involves strong checks on both government and commercial organizations. But to work with these ideas it would be necessary to look beyond Peter Thiel's entertaining antics and that is difficult because there is such delight in giving him a pedestal and spotlight. What is liberty? Talk to Peter Thiel. He must really know all about that.

ravenstine• 3 years ago

It's even what individuals do when they believe they can justify it. Many accident lawyers would be out of business if people refused to fake a "soft-tissue" injury out of principle. If the incentive is there, people will usually take it, if they don't have to feel badly about it.

acdha• 3 years ago

It’s important to remember that those libertarian views didn’t just happen by accident: a LOT of money had been spent developing media, organizations, even religious denominations starting with backlash to the New Deal. The historian Kevin Kruse wrote a book which will have a lot of familiar names:

http://kevinmkruse.com/book/one-nation-under-god/

JohnFen5• 3 years ago

If corporations are people, then they're sociopaths that society needs to be protected from.

comte70922• 3 years ago

Basic economic theory predicts that a in a market with perfect competition, all firms will have zero profits (the economic definition, wherein opportunity costs are taken into consideration, nominal profits aren’t zero).

It’s interesting to observe the number of people who will post about capitalism and competition and not consider the fact that for any rational actor, competition is existentially horrible and should be avoided by all means necessary. Perfect competition exists purely as a mathematical model and not as an actually observed outcome of a “natural” system for a reason.

nordsieck• 3 years ago

Basic economic theory predicts that a in a market with perfect competition, all firms will have zero profits (the economic definition, wherein opportunity costs are taken into consideration, nominal profits aren’t zero).

In your simplified economic world, I'm not convinced that profits would be pushed below the cost of capital.

kibwen• 3 years ago

Perfect competition is one end of a spectrum, where the other end is monopoly. Monopoly is a stable equilibrium, whereas perfect competition is not. On a long enough timeline, a perfectly competitive system will decay into a monopoly and stay there unless acted upon by an outside force designed to break up the monopoly.

michele_f• 3 years ago

By any chance, has Google being doing something different to improve and update its search engine?

hef19898• 3 years ago

If they did, it did everything but improving search results.

visarga• 3 years ago

Google is like old AltaVista and OpenAI like young Google. Around 1999, AV was bad at search and horribly full of spam and ads. Google was like chatGPT, it was strikingly better than anything before it, a clean interface and superior results.

How the wheel turns! They are captive to their own success. Time to move over and make space for someone who will innovate, Google can't find the way out of its local maximum, not even with all the employees and AIs put together.

roxgib• 3 years ago

Let me get this straight - they paid a company to scrape data for them, and then sued them for scraping their own data as well?

charcircuit• 3 years ago

That is oversimplified. They are against scraping things that against a site's terms.

"The collection of data from websites can serve legitimate integrity and commercial purposes, if done lawfully and in accordance with those websites’ terms"

lorey• 3 years ago

Roses are red, their logo is blue. They scrape your data, but sue if you do.

mdwalters• 3 years ago

very true

desi_ninja• 3 years ago

Hello, I am Nintendo. This is from my company policy book

shever732• 3 years ago

https://archive.ph/cRmtK

bvm• 3 years ago

scraper!

shever732• 3 years ago

The irony wasn’t lost on me :)

1vuio0pswjnm7• 3 years ago

This only comes to light because of litigation between Facebook and Bright Data.

Even when these "tech" companies settle to escape admitting to any wrongdoing and creating detrimental precedent, litigation can still be an effective way to reveal what nefarious practices are going on behind the scenes at these "tech" companies. Here, we learn that Facebook was paying scraping-for-hire providers for years while publicly condemning the practice of scraping.

If we knew the identities of the e-commerce sites being scraped perhaps the e-commerce companies could sue Facebook. Assuming Facebook would ignore a cease-and-desist letter.

r_singh3• 3 years ago

So does LinkedIn

bbkane2• 3 years ago

Could you point me to a link about how LinkedIn scrapes sites?

r_singh3• 3 years ago

I had a couple of bookmarks but weirdly the links are now broken and also not on Wayback machine.

https://www.nytimes.com/2011/07/10/technology/linkedin-is-ac...

https://readwrite.com/2011/07/11/linkedin-admits-to-data-scr...

primax2• 3 years ago

I'd love to see what kind of scraping tech Facebook use. I've been getting increasingly frustrated with the python scrapers and ecosystem and have been checking out Colly. But I'm sure whatever they have built must be nuts.

I wonder if you could train scrapers with AI with a reasonable level of success.

cldellow• 3 years ago

As someone who maintains a scraper [1] in the python ecosystem, can I ask what you find frustrating?

My scraper is targeted at smaller crawls, so your feedback might be totally irrelevant to me... but I always want to keep an ear out.

[1] https://github.com/cldellow/datasette-scraper/

primax2• 3 years ago

I haven't used your scraper, I worked with scrapy. I found debugging as well as command and control to be challenging. Colly was a big step up.

For reference I am primarily a Golang programmer so I find weakly typed tools frustrating.

imgabe3• 3 years ago

I don't consider this hypocritical at all. It's like getting mad at a soccer team. "Hey they were trying to score goals, but at the same time, preventing the other team from scoring goals! Hypocrites!"

ceejayoz• 3 years ago

If your soccer team is suing the other team claiming the rules say you're not allowed to try scoring goals, it's very much hypocrisy.

imgabe3• 3 years ago

If part of the rules of the game are that you’re allowed to sue people and lobby to change the rules, then no it’s very much not.

FlotaDeAragon2• 3 years ago

Facebook, Google don't produce anything

They collect data and reshape it and resell it

It's good for society but this activity should be fully legal for ALL

BbzzbB• 3 years ago

It basically is, but as with everything if your pockets are deep enough you can sue someone to bankruptcy for doing something they're allowed to. That issue is way deeper then scraping.

npteljes2• 3 years ago

What makes you say that they don't produce anything? They both maintain very popular platforms, so that people can communicate and organize. AND they do all the bullshit in the meantime.

FlotaDeAragon2• 3 years ago

They redistribute content others have produced. Filter and categorize it using automated processes

Exactly the same as ThePirateBay, conceptually

rib3ye• 3 years ago

Isn't making software "producing" something? Or do they need to make hardware like Apple?

randomdata• 3 years ago

I'm not sure hardware helps. All my Apple hardware does is collects data, reshapes it, and resells it.

spywaregorilla• 3 years ago

What kind of company produces things but not by collecting components and reshaping them?

aitchnyu• 3 years ago

Skeptical me wants to know if a judge proclaimed "do unto others..." in a judgement, or is likely to do so here.

napolux• 3 years ago

I know one thing or two about fb bot AKA scraper :P

https://news.ycombinator.com/item?id=23490367

Lucasoato• 3 years ago

Awesome read, the tactics you've adopted are incredible! Poor Eddie :)

type02• 3 years ago

I will never cease to be surprised how morally corrupt this company is. Hypocricity of FB/Meta is astounding.

marginalia_nu2• 3 years ago

Hate to be the voice of reason, but the size of Meta, I'm not even convinced whoever thought up this lawsuit was aware of the hypocrisy.

sorokod• 3 years ago

Too big to be honest?

marginalia_nu2• 3 years ago

Too big for the left hand to know what the right is doing.

moneywoes• 3 years ago

Has anyone been able to scrape Facebook effectively? Like marketplace for example? Been trying unsuccessfully

RobSm2• 3 years ago

amitrip• 3 years ago

I haven't tried, what's the issue?

scotty794• 3 years ago

After you climb up the ladder remember to pull the ladder up so noone can follow you up.

RobSm2• 3 years ago

Google scrapes everyones data yet you use Google every day and find it useful.

anononaut• 3 years ago

On this website more than others, you will find plenty of people who have dropped Google utterly. Not to mention their search engine is the poorest it has been in a decade. "Useful" isn't as accurate as "still left standing."

ok123456• 3 years ago

I remember when facebook wouldn't release an app for android in the Android 1.5 days because they were rueful that google disallowed them from scraping peoples contacts.

JohnFen5• 3 years ago

Facebook being shady?? Color me shocked!

NayamAmarshe• 3 years ago

The world would be a better place without Facebook.

phendrenad2• 3 years ago

It's too bad that we don't get more info than "Meta paid Bright Data to gather data from e-comerce sites in order to build brand profiles on Meta platforms".

What does that mean exactly? Were they scraping, say, Dell.com, in order to get the latest prices on Alienware laptops? Or were their bots spamming Middle-Class Joe's Bootstrapped e-Commerce site and putting him out of business?

leobg• 3 years ago

Not just Facebook. I was recently banned from NerdyData.com for scraping their data, even though my plan said “unlimited” downloads.

They said I can download as much as I want, but I’ll have to do it manually or pay extra for their API.

Luckily for them, the sites from which they source their data didn’t enforce such a rule, I suppose.

Meta was scraping sites for years while fighting the practice

Related Stories