X offices raided in France as UK opens fresh investigation into Grok

https://www.bbc.com/news/articles/ce3ex92557jo

by vikaveri • 6 months ago

596 630 6 months ago

bbc.com

pogue• 6 months ago

Finally, someone is taking action against the CSAM machine operating seemingly without penalty.

tjpnz• 6 months ago

It's also a massive problem on Meta. Hopefully this action isn't just a one-off.

direwolf2040• 6 months ago

Does Meta publish it themselves or is it user–generated?

robtherobber2• 6 months ago

The prosecutor's office also said it was leaving X and would communicate on LinkedIn and Instagram from now on.

I mean, perhaps it's time to completely drop these US-owned, closed-source, algo-driven controversial platforms, and start treating the communication with the public that funds your existence in different terms. The goal should be to reach as many people, of course, but also to ensure that the method and medium of communication is in the interest of the public at large.

spacecadet• 6 months ago

This. What a joke. Im still waiting on my tax refund from NYC for plastering "twitter" stickers on every publicly funded vehicle.

valar_m• 6 months ago

The goal should be to reach as many people, of course, but also to ensure that the method and medium of communication is in the interest of the public at large.

Who decides what communication is in the interest of the public at large? The Trump administration?

robtherobber2• 6 months ago

You appear to have posted a bit of a loaded question here, apologies if I'm misinterpreting your comment. It is, of course, the public that should decide what communication is of public interest, at least in a democracy operating optimally.

I suppose the answer, if we're serious about it, is somewhat more nuanced.

To begin, public administrations should not get to unilaterally define "the public interest" in their communication, nor should private platforms for that matter. Assuming we're still talking about a democracy, the decision-making should be democratically via a combination of law + rights + accountable institutions + public scrutiny, with implementation constraints that maximise reach, accessibility, auditability, and independence from private gatekeepers. The last bit is rather relevant, because the private sector's interests and the citizen's interests are nearly always at odds in any modern society, hence the state's roles as rule-setter (via democratic processes) and arbiter. Happy to get into further detail regarding the actual processes involved, if you're genuinely interested.

That aside - there are two separate problems that often get conflated when we talk about these platforms:

- one is reach: people are on Twitter, LinkedIn, Instagram, so publishing there increases distribution; public institutions should be interested in reaching as many citizens as possible with their comms;

- the other one is dependency: if those become the primary or exclusive channels, the state's relationship with citizens becomes contingent on private moderation, ranking algorithms, account lockouts, paywalls, data extraction, and opaque rule changes. That is entirely and dangerously misaligned with democratic accountability.

A potential middle position could be ti use commercial social platforms as secondary distribution instead of the authoritative channel, which in reality is often the case. However, due to the way societies work and how individuals operate within them, the public won't actually come across the information until it's distributed on the most popular platforms. Which is why some argue that they should be treated as public utilities since dominant communications infrastructure has quasi-public function (rest assured, I won't open that can of worms right now).

Politics is messy in practice, as all balancing acts are - a normal price to pay for any democratic society, I'd say. Mix that with technology, social psychology and philosophies of liberty, rights, and wellbeing, and you have a proper head-scratcher on your hands. We've already done a lot to balance these, for sure, but we're not there yet and it's a dynamic, developing field that presents new challenges.

Mordisquitos2• 6 months ago

I agree with you. In my opinion it was already bad enough that official institutions were using Twitter as a communication platform before it belonged to Musk and started to restrict visibility to non-logged in users, but at least Twitter was arguably a mostly open communication platform and could be misunderstood as a public service in the minds of the less well-informed. However, deciding to "communicate" at this day and age on LinkedIn and Instagram, neither of which ever made a passing attempt to pretend to be a public communications service, boggles the mind.

chrisjj21• 6 months ago

official institutions were using Twitter as a communication platform before it belonged to Musk and started to restrict visibility to non-logged in users

... thereby driving up adoption far better than Twitter itself could. Ironic or what.

nonethewiser• 6 months ago

I mean, perhaps it's time to completely drop these US-owned, closed-source, algo-driven controversial platforms

I think we are getting very close the the EU's own great firewall.

There is currently a sort of identity crisis in the regulation. Big tech companies are breaking the laws left and right. So which is it?

- fine harvesting mechanism? Keep as-is.

- true user protection? Blacklist.

lokar2• 6 months ago

Or the companies could obey the law

direwolf2040• 6 months ago

I support the EU harvesting money from evil companies

morkalork2• 6 months ago

In an ideal world they'd just have an RSS feed on their site and people, journalists, would subscribe to it. Voilà!

vessenes5• 6 months ago

Interesting. This is basically the second enforcement on speech / images that France has done - first was Pavel Durov @ Telegram. He eventually made changes in Telegram's moderation infrastructure and I think was allowed to leave France sometime last year.

I don't love heavy-handed enforcement on speech issues, but I do really like a heterogenous cultural situation, so I think it's interesting and probably to the overall good to have a country pushing on these matters very hard, just as a matter of keeping a diverse set of global standards, something that adds cultural resilience for humanity.

linkedin is not a replacement for twitter, though. I'm curious if they'll come back post-settlement.

derrida3• 6 months ago

I wouldn't equate the two.

There's someone who was being held responsible for what was in encrypted chats.

Then there's someone who published depictions of sexual abuse and minors.

Worlds apart.

direwolf2040• 6 months ago

Telegram isn't encrypted. For all the marketing about security, it has none, apart from TLS, and an optional "secret chat" feature that you have to explicitly select, only works with 2 participants and doesn't work very well.

They can read all messages, so they don't have an excuse for not helping in a criminal case. Their platform had a reputation of being safe for crime, which is because they just... ignored the police. Until they got arrested for that. They still turn a blind eye but not to the police.

btreecat2• 6 months ago

but I do really like a heterogenous cultural situation

Why isn't that a major red flag exactly?

vessenes5• 6 months ago

Hi there - author here. Care to add some specifics? I can imagine lots of complaints about this statement, but I don't know which (if any) you have.

tokai4• 6 months ago

In what world is generating CSAM a speech issue? Its really doing a disservice to actual free speech issues to frame it was such.

logicchains2• 6 months ago

The point of banning real CSAM is to stop the production of it, because the production is inherently harmful. The production of AI or human generated CSAM-like images does not inherently require the harm of children, so it's fundamentally a different consideration. That's why some countries, notably Japan, allow the production of hand-drawn material that in the US would be considered CSAM.

direwolf2040• 6 months ago

if pictures are speech, then either CSAM is speech, or you have to justify an exception to the general rule.

CSAM is banned speech.

logicchains2• 6 months ago

but I do really like a heterogenous cultural situation, so I think it's interesting and probably to the overall good to have a country pushing on these matters very hard

Censorship increases homogeneity, because it reduces the amount of ideas and opinions that are allowed to be expressed. The only resilience that comes from restricting people's speech is resilience of the people in power.

AureliusMA• 6 months ago

This is precisely the point of the comment you are replying to: a balance has to be found and enforced.

moolcool4• 6 months ago

I really don't see reasonable enforcement of CSAM laws as a restriction on "diversity of thought".

vessenes5• 6 months ago

You were downvoted -- a theme in this thread -- but I like what you're saying. I disagree, though, on a global scale. By resilience, I mean to reference something like a monoculture plantation vs a jungle. The monoculture plantation is vulnerable to anything that figures out how to attack it. In a jungle, a single plant or set might be vulnerable, but something that can attack all the plants is much harder to come by.

Humanity itself is trending more toward monoculture socially; I like a lot of things (and hate some) about the cultural trend. But what I like isn't very important, because I might be totally wrong in my likes; if only my likes dominated, the world would be a much less resilient place -- vulnerable to the weaknesses of whatever it is I like.

So, again, I propose for the race as a whole, broad cultural diversity is really critical, and worth protecting. Even if we really hate some of the forms it takes.

StopDisinfo9102• 6 months ago

Very different charges however.

Durov was held on suspicion Telegram was willingly failing to moderate its platform and allowed drug trafficking and other illegal activities to take place.

X has allegedly illegally sent data to the US in violation of GDPR and contributed to child porn distribution.

Note that both are directly related to direct violation of data safety law or association with a separate criminal activities, neither is about speech.

vessenes5• 6 months ago

I like your username, by the way.

CSAM was the lead in the 2024 news headlines in the French prosecution of Telegram also. I didn't follow the case enough to know where they went, or what the judge thought was credible.

From a US mindset, I'd say that generation of communication, including images, would fall under speech. But then we classify it very broadly here. Arranging drug deals on a messaging app definitely falls under the concept of speech in the US as well. Heck, I've been told by FBI agents that they believe assassination markets are legal in the US - protected speech.

Obviously, assassinations themselves, not so much.

pu_pe• 6 months ago

I suppose those are the offices from SpaceX now that they merged.

omnimus2• 6 months ago

So France is raiding offices of US military contractor?

mkjs• 6 months ago

How is that relevant? Are you implying that being a US military contractor should make you immune to the laws of other countries that you operate in?

The onus is on the contractor to make sure any classified information is kept securely. If by raiding an office in France a bunch of US military secrets are found, it would suggest the company is not fit to have those kind of contracts.

fanatic2pope• 6 months ago

Even if it is, being affiliated with the US military doesn't make you immune to local laws.

https://www.the-independent.com/news/world/americas/crime/us...

hermanzegerman• 6 months ago

I know it's hard to grasp for you. But in France, french laws and jurisdiction applies, not those of the United States

gitaarik• 6 months ago

So what?

afavour3• 6 months ago

I’m sure Musk is going to say this is about free speech in an attempt to gin up his supporters. It isn’t. It’s about generating and distributing non consensual sexual imagery, including of minors. And, when notified, doing nothing about it. If anything it should be an embarrassment that France are the only ones doing this.

(it’ll be interesting to see if this discussion is allowed on HN. Almost every other discussion on this topic has been flagged…)

cbeach• 6 months ago

when notified, doing nothing about it

When notified, he immediately:

  * "implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing" - https://www.bbc.co.uk/news/articles/ce8gz8g2qnlo 

  * locked image generation down to paid accounts only (i.e. those individuals that can be identified via their payment details).

Have the other AI companies followed suit? They were also allowing users to undress real people, but it seems the media is ignoring that and focussing their ire only on Musk's companies...

derrida3• 6 months ago

The other LLMs probably don't have the training data in the first place.

afavour3• 6 months ago

You and I must have different definitions of the word “immediately”. The article you posted is from January 15th. Here is a story from January 2nd:

https://www.bbc.com/news/articles/c98p1r4e6m8o

Have the other AI companies followed suit? They were also allowing users to undress real people

No they weren’t? There were numerous examples of people feeding the same prompts to different AIs and having their requests refused. Not to mention, X was also publicly distributing that material, something other AI companies were not doing. Which is an entirely different legal liability.

freejazz4• 6 months ago

Kiddie porn but only for the paying accounts!

rsynnott3• 6 months ago

If anything it should be an embarrassment that France are the only ones doing this.

As mentioned in the article, the UK's ICO and the EC are also investigating.

France is notably keen on raids for this sort of thing, and a lot of things that would be basically a desk investigation in other countries result in a raid in France.

chrisjj21• 6 months ago

Full marks to France for addressing its higher than average rate of unemployment.

techblueberry2• 6 months ago

I'm not saying I'm entirely against this, but just out of curiosity, what do they hope to find in a raid of the french offices, a folder labeled "Grok's CSAM Plan"?

afavour3• 6 months ago

It was known that Grok was generating these images long before any action was taken. I imagine they’ll be looking for internal communications on what they were doing, or deciding not to do, doing during that time.

Mordisquitos2• 6 months ago

What do they hope to find, specifically? Who knows, but maybe the prosecutors have a better awareness of specifics than us HN commenters who have not been involved in the investigation.

What may they find, hypothetically? Who knows, but maybe an internal email saying, for instance, 'Management says keep the nude photo functionality, just hide it behind a feature flag', or maybe 'Great idea to keep a backup of the images, but must cover our tracks', or perhaps 'Elon says no action on Grok nude images, we are officially unaware anything is happening.'

cwillu6• 6 months ago

Or “regulators don't understand the technology; short of turning it off entirely, there's nothing we can do to prevent it entirely, and the costs involved in attempting to reduce it are much greater than the likely fine, especially given that we're likely to receive such a fine anyway.”

rsynnott3• 6 months ago

what do they hope to find in a raid of the french offices, a folder labeled "Grok's CSAM Plan"?

You would be _amazed_ at the things that people commit to email and similar.

Here's a Facebook one (leaked, not extracted by authorities): https://www.reuters.com/investigates/special-report/meta-ai-...

plopilop• 6 months ago

I mean, the example you link is probably an engineer doing their job of signalling to hierarchy that something went deeply wrong. Of course, the lack of action of Facebook afterwards is a proof that they did not care, but not as much as a smoking gun.

A smoking gun would be, for instance, Facebook observing that most of their ads are scam, that the cost of fixing this exceeds by far "the cost of any regulatory settlement involving scam ads.", and to conclude that the company’s leadership decided to act only in response to impending regulatory action.

https://www.reuters.com/investigations/meta-is-earning-fortu...

moolcool4• 6 months ago

Moderation rules? Training data? Abuse metrics? Identities of users who generated or accessed CSAM?

bryan_w• 6 months ago

Do you think that data is stored at the office? Where do you think the data is stored? The janitors closet?

reaperducer• 6 months ago

out of curiosity, what do they hope to find in a raid of the french offices, a folder labeled "Grok's CSAM Plan"?

You're not too far off.

There was a good article in the Washington Post yesterday about many many people inside the company raising alarms about the content and its legal risk, but they were blown off by managers chasing engagement metrics. They even made up a whole new metric.

There was also prompts telling the AI to act angry or sexy or other things just to keep users addicted.

arppacket• 6 months ago

There was a WaPo article yesterday, that talked about how xAI deliberately loosened Grok’s safety guardrails and relaxed restrictions on sexual content in an effort to make the chatbot more engaging and “sticky” for users. xAI employees had to sign new waivers in the summer, and start working with harmful content, in order to train and enable those features.

I assume the raid is hoping to find communications to establish that timeline, maybe internal concerns that were ignored? Also internal metrics that might show they were aware of the problem. External analysts said Grok was generating a CSAM image every minute!!

https://www.washingtonpost.com/technology/2026/02/02/elon-mu...

chrisjj21• 6 months ago

External analysts said Grok was generating a CSAM image every minute!!
https://www.washingtonpost.com/technology/2026/02/02/elon-mu...

That article has no mention of CSAM. As expected, since you can bet the Post has lawyers checking.

direwolf2040• 6 months ago

Maybe emails between the French office and the head office warning they may violate laws, and the response by head office?

pjc507• 6 months ago

Since the release of (some of) the Epstein files, that kind of "let's do some crimes" email seems much more plausible.

bluegatty2• 6 months ago

Email history caches. They could also have provided requirements to provide communications etc..

kmeisthax• 6 months ago

Have you taken a look at the Epstein files lately? Rich people write out basically all of their crimes in triplicate because they don't fear the law.

Altern4tiveAcc3• 6 months ago

Prosecutors say they are now investigating whether X has broken the law across multiple areas.

This step could come before a police raid.

This looks like plain political pressure. No lives were saved, and no crime was prevented by harassing local workers.

moolcool4• 6 months ago

This looks like plain political pressure. No lives were saved, and no crime was prevented by harassing local workers.

The company made and released a tool with seemingly no guard-rails, which was used en masse to generate deepfakes and child pornography.

trhway3• 6 months ago

Internet routers, network cards, the computers, OS and various application software have no guardrails and is used for all the nefarious things. Why those companies aren't raided?

pdpi5• 6 months ago

I'm of two minds about this.

One the one hand, it seems "obvious" that Grok should somehow be legally required to have guardrails stopping it from producing kiddie porn.

On the other hand, it also seems "obvious" that laws forcing 3D printers to detect and block attempts to print firearms are patently bullshit.

The thing is, I'm not sure how I can reconcile those two seemingly-obvious statements in a principled manner.

cubefox4• 6 months ago

The company made and released a tool with seemingly no guard-rails, which was used en masse to generate deepfakes and child pornography.

Do you have any evidence for that? As far as I can tell, this is false. The only thing I saw was Grok changing photos of adults into them wearing bikinis, which is far less bad.

ljsprague• 6 months ago

No other "AI" companies released tools that could do the same?

aaomidi• 6 months ago

Lmao they literally made a broad accessible CSAM maker.

orwin• 6 months ago

France prosecutors use police raids way more than other western countries. Banks, political parties, ex-presidents, corporate HQs, worksites... Here, while white-collar crimes are punished as much as in the US (i.e very little), we do at least investigate them.

bawolff2• 6 months ago

and no crime was prevented by harassing local workers.

Siezing records is usually a major step in an investigation. Its how you get evidence.

Sure it could just be harrasment, but this is also how normal police work looks. France has a reasonable judicial system so absent of other evidence i'm inclined to believe this was legit.

giancarlostoro• 6 months ago

This looks like plain political pressure. No lives were saved, and no crime was prevented by harassing local workers.

I wouldn't even consider this a reason if it wasn't for the fact that OpenAI and Google, and hell literally every image model out there all have the same "this guy edited this underage girls face into a bikini" problem (this was the most public example I've heard so I'm going with that as my example). People still jailbreak chatgpt, and they've poured how much money into that?

emsign2• 6 months ago

They've already broken the law by creating and hosting CSAM. Now let's see what else prosecutors will find.

317070• 6 months ago

Well, there is evidence that this company made and distributed CSAM and pornographic deepfakes to make a profit. There is no evidence lacking there for the investigators.

So the question becomes if it was done knowingly or recklessly, hence a police raid for evidence.

See also [0] for a legal discussion in the German context.

[0] https://arxiv.org/html/2601.03788v1

skissane2• 6 months ago

Well, there is evidence that this company made and distributed CSAM

I think one big issue with this statement – "CSAM" lacks a precise legal definition; the precise legal term(s) vary from country to country, with differing definitions. While sexual imagery of real minors is highly illegal everywhere, there's a whole lot of other material – textual stories, drawings, animation, AI-generated images of nonexistent minors – which can be extremely criminal on one side of an international border, de facto legal on the other.

And I'm not actually sure what the legal definition is in France; the relevant article of the French Penal Code 227-23 [0] seems superficially similar to the legal definition of "child pornography" in the United States (post-Ashcroft vs Free Speech Coalition), and so some–but (maybe) not all–of the "CSAM" Grok is accused of generating wouldn't actually fall under it. (But of course, I don't know how French courts interpret it, so maybe what it means in practice is something broader than my reading of the text suggests.)

And I think this is part of the issue – xAI's executives are likely focused on compliance with US law on these topics, less concerned with complying with non-US law, in spite of the fact that CSAM laws in much of the rest of the world are much broader than in the US. That's less of an issue for Anthropic/Google/OpenAI, since their executives don't have the same "anything that's legal" attitude which xAI often has. And, as I said – while that's undoubtedly true in general, I'm unsure to what extent it is actually true for France in particular.

[0] https://www.legifrance.gouv.fr/codes/section_lc/LEGITEXT0000...

bluegatty2• 6 months ago

No, that's not at all how this works.

They have a court order obviously to collect evidence.

You have offered zero evidence to indicate there is 'political pressure' and that statement by prosecutors doesn't hint at that.

'No crime was prevented by harassing workers' is essentially non sequitor in this context.

It could be that that this is political nonsense, but there would have to be more details.

These issues are really hard but we have to confront them. X can alter electoral outcomes. That's where we are at.

stickfigure2• 6 months ago

Honest question: What does it mean to "raid" the offices of a tech company? It's not like they have file cabinets with paper records. Are they just seizing employee workstations?

Seems like you'd want to subpoena source code or gmail history or something like that. Not much interesting in an office these days.

beart• 6 months ago

Offline syncing of outlook could reveal a lot of emails that would otherwise be on a foreign server. A lot of people save copies of documents locally as well.

cm21872• 6 months ago

Most enterprises have fully encrypted workstations, when they don't use VM where the desktop is just a thin client that doesn't store any data. So there should be really nothing of interest in the office itself.

bsimpson• 6 months ago

I had the same thought - not just about raids, but about raiding a satellite office. This sounds like theater begging for headlines like this one.

direwolf2040• 6 months ago

They do what they can. They obviously can't raid the American office.

paxys• 6 months ago

Whether you are a tech company or not, there's a lot of data on computers that are physically in the office.

ramuel2• 6 months ago

Except when they have encryption, which should be the standard? I mean how much data would authorities actually retrieve when most stuff is located on X servers anyways? I have my doubts.

ronsor2• 6 months ago

These days many tech company offices have a "panic button" for raids that will erase data. Uber is perhaps the most notorious example.

politelemon• 6 months ago

It's sad to see this degree of incentives perverted, over adhering to local laws.

caminante5• 6 months ago

notorious

What happened to due process? Every major firm should have a "dawn raid" policy to comply while preserving rights.

Specific to the Uber case(s), if it were illegal, then why didn't Uber get criminal charges or fines?

At best there's an argument that it was "obstructing justice," but logging people off, encrypting, and deleting local copies isn't necessarily illegal.

BrandoElFollito5• 6 months ago

This is a perfect way for the legal head of the company in-country to visit some jails.

They will explain that it was done remotely and whatnot but then the company will be closed in the country. Whether this matters for the mothership is another story.

mr_mitm• 6 months ago

How do you know this?

wasabi991011• 6 months ago

It wasn't erasing as far I know, but locking all computers.

Covered here: https://www.theguardian.com/news/2022/jul/10/uber-bosses-tol...

digiown• 6 months ago

Or they just connect to a mothership with keys on the machine. The authorities can have the keys, but alas, they're useless now, because there is some employee watching the surveillance cameras in the US, and he pressed a red button revoking all of them. What part of this is illegal?

Obviously, the government can just threaten to fine you any amount, close operations or whatever, but your company can just decide to stop operating there, like Google after Russia imposed an absurd fine.

alex11383• 6 months ago

Why is this the most upvoted question? Obsessing over pedantry rather than the main thrust of what's being discussed

KaiserPro5• 6 months ago

Gather evidence.

I assume that they have opened a formal investigation and are now going to the office to collect/perloin evidence before it's destroyed.

Most FAANG companies have training specifically for this. I assume X doesn't anymore, because they are cool and edgy, and staff training is for the woke.

niemandhier4• 6 months ago

If that training involves destroying evidence or withholding evidence from the prosecution, you are going to jail if you follow it.

niemandhier4• 6 months ago

Gather evidence against employees, use that evidence to put them under pressure to testify against their employer or grant access to evidence.

Sabu was put under pressure by the FBI, they threatened to place his kids into foster care.

That was legal. Guess what, similar things would be legal in France.

We all forget that money is nice, but nation states have real power. Western liberal democracies just rarely use it.

The same way the president of the USA can order a Drone strike on a Taliban war lord, the president of France could order Musks plane to be escorted to Paris by 3 Fighter jets.

mmooss• 6 months ago

Western liberal democracies just rarely use it.

Also, they are restricted in how they use it, and defendents have rights and due process.

Sabu was put under pressure by the FBI, they threatened to place his kids into foster care.

Though things like that can happen, which are very serious.

kps• 6 months ago

We all forget that money is nice, but nation states have real power.

Elon has ICBMs, but France has warheads.

ChrisMarshallNY2• 6 months ago

We all forget that money is nice, but nation states have real power.

I remember something (probably linked from here), where the essayist was comparing Jack Ma, one of the richest men on earth, and Xi Jinping, a much lower-paid individual.

They indicated that Xi got Ma into a chokehold. I think he "disappeared" Ma for some time. Don't remember exactly how long, but it may have been over a year.

hiprob• 6 months ago

It's legal to just put kids in foster care for no reason but to ruin someone's life?

gruez3• 6 months ago

Sabu was put under pressure by the FBI, they threatened to place his kids into foster care.
That was legal. Guess what, similar things would be legal in France.

lawfare is... good now? Between Trump being hit with felony charges for falsifying business records (lawfare is good?) and Lisa Cook getting prosecuted for mortgage fraud (lawfare is bad?), I honestly lost track at this point.

The same way the president of the USA can order a Drone strike on a Taliban war lord, the president of France could order Musks plane to be escorted to Paris by 3 Fighter jets.

What's even the implication here? That they're going to shoot his plane down? If there's no threat of violence, what does the French government even hope to achieve with this?

xoxolian• 6 months ago

We all forget that money is nice, but nation states have real power.

Interesting point. There's a top gangster who can buy anything in the prison commissary; and then there's the warden.

SpaceManNabs• 6 months ago

Sabu was put under pressure by the FBI, they threatened to place his kids into foster care.

This is pretty messed up btw.

Social work for children systems in the USA are very messed up. It is not uncommon for minority families to lose rights to parent their children for very innocuous things that would not happen to a non-oppressed class.

It is just another way for the justice/legal system to pressure families that have not been convicted / penalized under the supervision of a court.

And this isn't the only lever they use.

Every time I read crap like this I just think of Aaron Swartz.

projektfu• 6 months ago

Wait, Sabu's kids were foster kids. He was fostering them. Certainly if he went to jail, they'd go back to the system.

I mean, if you're a sole caretaker and you've been arrested for a crime, and the evidence looks like you'll go to prison, you're going to have to decide what to do with the care of your kids on your mind. I suppose that would pressure you to become an informant instead of taking a longer prison sentence, but there's pressure to do that anyway, like not wanting to be in prison for a long time.

cadamsdotcom• 6 months ago

Yes but using such power unscrupulously is a great way to lose it.

chrisjj21• 6 months ago

Gather evidence against employees

I'm sure they have much better and quieter ways to do that.

Whereas a raid is #1 choice for max volume...

Aurornis2• 6 months ago

Seems like you'd want to subpoena source code or gmail history or something like that.

This would be done in parallel for key sources.

There is a lot of information on physical devices that is helpful, though. Even discovering additional apps and services used on the devices can lead to more discovery via those cloud services, if relevant.

Physical devices have a lot of additional information, though: Files people are actively working on, saved snippets and screenshots of important conversations, and synced data that might be easier to get offline than through legal means against the providers.

In outright criminal cases it's not uncommon for individuals to keep extra information on their laptop, phone, or a USB drive hidden in their office as an insurance policy.

This is yet another good reason to keep your work and personal devices separate, as hard as that can be at times. If there's a lawsuit you don't want your personal laptop and phone to disappear for a while.

charcircuit• 6 months ago

Sure it might be on the device, but they would need a password to decrypt the laptop's storage to get any of the data. There's also the possibility of the MDM software making it impossible to decrypt if given a remote signal. Even if you image the drive, you can't image the secure enclave so if it is wiped it's impossible to retrieve.

nebula88042• 6 months ago

I read somewhere that Musk (or maybe Theil) companies have processes in place to quickly offload data from a location to other jurisdictions (and destroy the local data) when they detect a raid happening. Don't know how true it is though. The only insight I have into their operations was the amazing speed by which people are badged in and out of his various gigafactories. It "appears" that they developed custom badging systems when people drive into gigafactories to cut the time needed to begin work. If they are doing that kind of stuff then there has got to be something in place for a raid. (This is second hand so take with a grain of salt)

EDIT: It seems from other comments that it may have been Uber I was reading about. The badging system I have personally observed outside the Gigafactories. Apologies for the mixup.

malfist2• 6 months ago

That is very much illegal in the US

aucisson_masque• 6 months ago

Are they just seizing employee workstations?

Yes.

jimbo808• 6 months ago

It sounds better in the news when you do a raid. These things are generally not done for any purpose other than to communicate a message and score political points.

eli• 6 months ago

Why don't you think they have file cabinets and paper records?

ChuckMcM3• 6 months ago

Sadly the media calls the lawful use of a warrant a 'raid' but that's another issue.

The warrant will have detailed what it is they are looking for, French warrants (and legal system!) are quite a bit different than the US but in broad terms operate similarly. It suggests that an enforcement agency believes that there is evidence of a crime at the offices.

As a former IT/operations guy I'd guess they want on-prem servers with things like email and shared storage, stuff that would hold internal discussions about the thing they were interested in, but that is just my guess based on the article saying this is related to the earlier complaint that Grok was generating CSAM on demand.

chrisjj21• 6 months ago

I'd guess they want on-prem servers with things like email and shared storage

For a net company in 2026? Fat chance.

vintermann5• 6 months ago

It is a raid in that it's not expected, it relies on not being expected, and they come and take away your stuff by force. Maybe it's a legal raid, but let's not sugar coat it, it's still a raid and whether you're guilty or not it will cause you a lot of problems.

terminalshort6• 6 months ago

Who has on prem servers at an office location?

anigbrowl8• 6 months ago

They do have some physical records, but it would be mostly investigators producing a warrant and forcing staff to hand over administrative credentials to allow forensic data collection.

chrisjj21• 6 months ago

forcing staff to hand over administrative credentials to allow forensic data collection.

What, thinking HQ wouldn't cancel them?

direwolf2040• 6 months ago

Usually they steal all electronic devices.

verdverm5• 6 months ago

France24 article on this: https://www.france24.com/en/france/20260203-paris-prosecutor...

lol, they summoned Elon for a hearing on 420

"Summons for voluntary interviews on April 20, 2026, in Paris have been sent to Mr. Elon Musk and Ms. Linda Yaccarino, in their capacity as de facto and de jure managers of the X platform at the time of the events,

miltonlost2• 6 months ago

I wonder how he'll try to get out of being summoned. Claim 4/20 is a holiday that he celebrates?

verdverm5• 6 months ago

It's voluntary

flohofwoe• 6 months ago

Claim 4/20 is a holiday that he celebrates?

Given his recent "far right" bromance that's probably not a good idea ;)

bean469• 6 months ago

Claim 4/20 is a holiday that he celebrates?

Most likely, it's Hitler's birthday after all

BrandoElFollito5• 6 months ago

Why "lol"?

verdverm5• 6 months ago

420 is a stoner number, stoners lol a lot, thought of Elmo's failed joint smoking on JRE before I stopped watching

...but then other commenters reminded me there is another thing on the same date, which might have been more the actual troll at Elmo to get him all worked up

why_at• 6 months ago

The Paris prosecutor's office said it launched the investigation after being contacted by a lawmaker alleging that biased algorithms in X were likely to have distorted the operation of an automated data processing system.

I'm not at all familiar with French law, and I don't have any sympathy for Elon Musk or X. That said, is this a crime?

Distorted the operation how? By making their chatbot more likely to say stupid conspiracies or something? Is that even against the law?

int_19h2• 6 months ago

Holocaust denial is illegal in France, for one, and Grok did exactly that on several occasions.

mschuster913• 6 months ago

I'm not at all familiar with French law, and I don't have any sympathy for Elon Musk or X. That said, is this a crime?

GDPR and DMA actually have teeth. They just haven't been shown yet because the usual M.O. for European law violators is first, a free reminder "hey guys, what you're doing is against the law, stop it, or else". Then, if violations continue, maybe two or three rounds follow... but at some point, especially if the violations are openly intentional (and Musk's behavior makes that very very clear), the hammer gets brought down.

Our system is based on the idea that we institute complex regulations, and when they get introduced and stuff goes south, we assume that it's innocent mistakes first.

And in addition to that, there's the geopolitical aspect... basically, hurt Musk to show Trump that, yes, Europe means business and has the means to fight back.

As for the allegations:

The probe has since expanded to investigate alleged “complicity” in spreading pornographic images of minors, sexually explicit deepfakes, denial of crimes against humanity and manipulation of an automated data processing system as part of an organised group, and other offences, the office said in a statement Tuesday.

The GDPR/DMA stuff just was the opener anyway. CSAM isn't liked by authorities at all, and genocide denial (we're not talking about Palestine here, calm your horses y'all, we're talking about Holocaust denial) is a crime in most European jurisdiction (in addition to doing the right-arm salute and other displays of fascist insignia). We actually learned something out of WW2.

direwolf2040• 6 months ago

GDPR has some stuff about biased algorithms. It's all civil, of course, no prison time for that, just fines.

xdennis• 6 months ago

lol, they summoned Elon for a hearing on 420

No. It's 20 April in the rest of the world: 204.

r721• 6 months ago

Another discussion: https://news.ycombinator.com/item?id=46872894

scotty79• 6 months ago

Facebook offices should routinely raided for aiding and profitting from various scams propagated through ads on this platform.

DaSHacka2• 6 months ago

That would apply to any and all social media though

ToucanLoucan3• 6 months ago

Sounds awesome, when do we start?

bluescrn2• 6 months ago

Governments don't care about minor scams. Political speech against them, on the other hand...

mkoubaa• 6 months ago

Governments prosecute violations of laws in ways that suit their interest. News at 11

ta9000• 6 months ago

Guess that will be a SpaceX problem soon enough. What a mess.

nebula88042• 6 months ago

I wonder if the recent announcement spurred them into making a move now rather than later.

tyre3• 6 months ago

The merger was most likely now because they have to do it before the IPO. After the IPO, there’s a whole process to force independent evaluation and negotiation between two boards / executives, which would be an absolute dumpster fire where Musk controls both.

When they’re both private, fine, whatever.

mschuster913• 6 months ago

How was that move legal anyway? Like... a lot of people and governments gave Musk money to develop, build and launch rockets. And now he's using it to bail out his failing social media network and CSAM peddling AI service.

wmf• 6 months ago

Once he launched the rockets he can do whatever he wants with the profit. And he wants to train Grok.

Psillisp• 6 months ago

CSAM in space! At least he isn’t reinventing the cross town bus.

piekvorst• 6 months ago

Yet another nail

tomlockwood2• 6 months ago

Elon's in the files asking Epstein about "wild parties" and then doesn't seem to care about all this. Easy to draw a conclusion here.

alex11383• 6 months ago

Elon is literally in the files, talking about going to the island. It's documented

chihuahua• 6 months ago

He was only going to the island to get rid of bots on Twitter. Just like OJ spent the rest of his life looking for the real killer.

yodsanklai• 6 months ago

Who knows who did what on this island, and I hope we'll figure it out. But in the meantime, going to this island or/and being friend with Epstein doesn't automatically make someone a pedo or rapist.

TZubiri• 6 months ago

Why would X have offices in France? I'm assuming it's just to hire French workers? Probably leftover from the Pre Acquisition era.

Or is there any France-specific compliance that must be done in order to operate in that country?

mike-the-mikado• 6 months ago

X makes its money selling advertising. France is the obvious place to have an office selling advertising to a large European French-speaking audience.

joshuaheard• 6 months ago

Yes, Paris is an international capital and centrally located for Europe, the Middle East, and Africa. Many tech companies have sales offices there.

tehjoker• 6 months ago

It's cool that not every law enforcement agency in the world is under the complete thumb of U.S. based billionaires.

ChrisMarshallNY2• 6 months ago

They have also summoned billionaire owner Elon Musk for questioning.

Good luck with that...

dathinab• 6 months ago

the thing is a lot of recent legal preceding surrounding X is about weather X fulfilled the legally required due diligence and if not what level of negligence we are speaking about

and the things about negligence which caused harm to humans (instead of e.g. just financial harm) is that

a) you can't opt out of responsibility, it doesn't matter what you put into your TOS or other contracts

b) executives which are found responsible for the negligent action of a company can be hold _personally_ liable

and independent of what X actually did Musk as highest level executive personal did

1) frequently did statements that imply gross negligence (to be clear that isn't necessary how X acted, which is the actual relevant part)

2) claimed that all major engineering decisions etc. are from him and no one else (because he love bragging about how good of an engineer he is)

This means summoning him for questioning is legally speaking a must have independent of weather you expect him to show up or not. And he probably should take it serious, even if that just means he also could send a different higher level executive from X instead.

sleepybrett4• 6 months ago

I guess he could just never enter the EU ever again. Maybe he can buy Little St. James.

a_better_world• 6 months ago

and count on Trump to disrespect the extradition treaties. Which might be a reasonable expectation, but will have consequences, and Trump might not hold power forever.

sleepybrett4• 6 months ago

I guess this means that building the neverending 'deepfake CSAM on demand machine' was a bad idea.

sleepybrett4• 6 months ago

I feel like whoever downvoted this comment should probably be on a watchlist of some kind.

jongjong• 6 months ago

Once you've worked long enough in the software industry, you start to understand it's all just a fully planned economy.

lukasm• 6 months ago

This is a show of resolve.

"Uh guys, little heads up: there are some agents of federal law enforcement raiding the premises, so if you see that. That’s what that is."

justaboutanyone2• 6 months ago

This sort of thing will be great for the SpaceX IPO :/

stubish2• 6 months ago

Especially if contracts with SpaceX start being torn up because the various ongoing investigations and prosecutions of xAI are now ongoing investigations and prosecutions of SpaceX. And next new lawsuits for creating this conflict of interest by merger.

darepublic• 6 months ago

I remember encountering questionable hentai material (by accident) back in the Twitter days. But back then twitter was a leftist darling

nemomarx• 6 months ago

I think there's a difference between "user uploaded material isn't properly moderated" and "the sites own chatbot generates porn on request based on images of women who didn't agree to it", no?

nailer• 6 months ago

But it doesn’t. Group has always had Aggressive filters on sexual content just like every other generative AI tool.

People who have found exploits, just like other generative AI tool.

tick_tock_tick2• 6 months ago

Not really?

techblueberry2• 6 months ago

Did you report it or just let it continue doing harm?

fumar• 6 months ago

Define leftist for back in the twitter days? I used twitter early in release. Don’t recall it being a faction specific platform.

reverius423• 6 months ago

I think they're using it in the American sense, which means "anywhere in the political spectrum of the leftmost 60% of the population".

direwolf2040• 6 months ago

Hentai has different legal status to realistic pictures of real people

isodev• 6 months ago

Good and honestly it’s high time. There used to be a time when we could give corps the benefit of the doubt but that time is clearly over. Beyond the CSAM, X is a cesspool of misinformation and generally the worst examples of humanity.

mekdoonggi5• 6 months ago

I don't know how we even allowed these companies to not be responsible for everything on their platform.

The entirety of the social media platform is based on the idea that the company isn't responsible for what the users post, which is just wrong. If you own a magazine, you should be held responsible for everything published.

You shouldn't be allowed to profit from publishing anything, then hide behind "the users did it, not us".

And in this case, Elon should be held responsible for every single image of CSAM published on X. Same with Zuck. Same with Truth Social, whatever you want.

miki123211• 6 months ago

This vindicates the pro-AI censorship crowd I guess.

It definitely makes it clear what is expected of AI companies. Your users aren't responsible for what they use your model for, you are, so you'd better make sure your model can't ever be used for anything nefarious. If you can't do that without keeping the model closed and verifying everyone's identities... well, that's good for your profits I guess.

culi• 6 months ago

It's not really different from how we treat any other platform that can host CSAM. I guess the main difference is that it's being "made" instead of simply "distributed" here

themafia2• 6 months ago

Holding corporations accountable for their profit streams is "censorship?" I wish they'd stop passing models trained on internet conversations and hoarded data as fit for any purpose. The world does not need to boil oceans for hallucinating chat bots at this particular point in history.

themafia2• 6 months ago

What would be censorship is if those same companies then brigaded forums and interfered with conversations and votes in an effort to try to hide their greed and criminality.

Not that this would _ever_ happen on Hacker News. :|

Jordan-117• 6 months ago

I could maybe see this argument if we were talking about raiding Stable Diffusion or Facebook or some other provider of local models. But the content at issue was generated not just by Twitter's AI model, but on their servers, integrated directly into their UI and hosted publicly on their platform. That makes them much more clearly culpable -- they're not just enabling this shit, they're creating it themselves on demand (and posting it directly to victims' public profiles).

disgruntledphd2• 6 months ago

And importantly, this is clearly published by Grok, rather than the user, so in this case (obviously this isn't the US) but if it was I'm not sure Section 230 would apply.

popalchemist2• 6 months ago

It's a bit of a leap to say that the model must be censored. SD and all the open image gen models are capable of all kinds of things, but nobody has gone after the open model trainers. They have gone after the companies making profits from providing services.

vintermann5• 6 months ago

So far, yes, but as far as I can tell their case against the AI giants aren't based on it being for-profit services in any way.

KaiserPro5• 6 months ago

Again its all about reasonable.

Firstly does the open model explicitly/tacitly allow CSAM generation?

Secondly, when the trainers are made aware of the problem, do they ignore it or attempt to put in place protections?

Thirdly, do they pull in data that is likely to allow that kind of content to be generated?

Fourthly, when they are told that this is happening, do they pull the model?

Fithly, do they charge for access/host the service and allow users to generate said content on their own servers?

mnewme7• 6 months ago

This is the wrong take.

Yes they could have an uncensored model, but then they would need proper moderation and delete this kind of content instantly or ban users that produce it. Or don’t allow it in the first place.

It doesn’t matter how CSAM is produced, the only thing that matters is that is on the platform.

I am flabbergasted people even defend this

direwolf2040• 6 months ago

It matters whether they attempt to block it or encourage it. Musk encouraged it, until legal pressure hit, then moved it behind a paywall so it's harder to see evidence.

madeofpalk• 6 months ago

Let’s take a step back and remove AI generation from the conversation for a moment.

Did X do enough to prevent its website being used to distribute illegal content - consensual sexual material of both adults and children?

Now reintroduce AI generation, where X plays a more active role in facilitating the creation of that illegal content.

gordian-mind3• 6 months ago

This is not about AI but about censorship of a nonaligned social network. It's been a developing current in EU. They have basically been foaming at the mouth at the platform since it got bought.

direwolf2040• 6 months ago

It's about a guy who thinks posting child porn on twitter is hilarious and that guy happens to own twitter.

If it was about blocking the social media they'd just block it, like they did with Russia Today, CUII-Liste Lina, or Pavel Durov.

direwolf2040• 6 months ago

It's not because it could generate CSAM. It's because when they found out it could generate CSAM, they didn't try to prevent that, they advertised it. Actual knowledge is a required compenent of many crimes.

bhelkey• 6 months ago

when they found out it could generate CSAM, they didn't try to prevent that, they advertised it.

Twitter publicly advertised it can create CSAM?

I have been off twitter for several years and I am open to being wrong here but that sounds unlikely.

code_for_monkey3• 6 months ago

I think having guardrails on your AI to not be able to produce this stuff is good actually. Also, Elon encourages this behavior socially through his posts so yeah he should face consequences.

keepamovin• 6 months ago

There's no crowds or sides. It's all manufactured divisions because some of those who can't or don't want to create the technology are determined to control it. So they'll get you mad about what they need to, to justify actions that increase their control.

It's the same playbook that is used again and again. For war, civil liberties crackdowns, lockdowns, COVID, etc, etc: 0) I want (1); start playbook: A) Something bad is here, B) You need to feel X + Panic about it, C) We are solving it via (1). Because you reacted at B, you will support C. Problem, reaction, solution. Gives the playmakers the (1) they want.

We all know this is going on. But I guess we like knowing someone is pulling the strings. We like being led and maybe even manipulated because perhaps in the familiar system (which yields the undeniable goods of our current way of life), there is safety and stability? How else to explain.

Maybe the need to be entertained with drama is a hackable side effect of stable societies populated by people who evolved as warriors, hunters and survivors.

commandlinefan• 6 months ago

Pretty disturbing to me how many people _on here_ are cheering for this. I thought that at least here of all places, there might be some nuanced discussion on "ok, I see why people are emotional about this topic in particular, but it's worth stepping back and putting emotions aside for a minute to see if this is actually reasonable overall..." but besides your comment, I'm not seeing much of that.

troyvit• 6 months ago

There's pro-AI censorship and then there's pro-social media censorship. It was the X offices that were raided. X is a social media company. They would have been raided whether it was AI that created the CSAM or a bunch of X contractors generating it mechanical-turk style.

I think the HN crowd is more nuanced than you're giving them credit for: https://hn.algolia.com/?q=chat+control

utopiah4• 6 months ago

To people claiming a physical raid is pointless from the point of gathering data :

- you are thinking about a company doing good things the right way. You are thinking about a company abiding by the law, storing data on its own server, having good practices, etc.

The moment a company starts to do dubious stuff then good practices start to go out the window. People write email with cryptic analogies, people start deleting emails, ... then as the circumvention become more numerous and complex, there needs to still be a trail in order to remain understandable. That trail will be in written form somehow and that must be hidden. It might be paper, it might be shadow IT but the point is that if you are not just forgetting to keep track of coffee pods at the social corner, you will leave traces.

So yes, raids do make sense BECAUSE it's about recurring complex activities that are just too hard to keep in the mind of one single individual over long periods of time.

tick_tock_tick2• 6 months ago

No need to be coy the raid exists because it's a way to punish the company without proving anything. They have zero intention of getting even the slightest bit of valuable data related to Grok from this.

direwolf2040• 6 months ago

What's your evidence?

fyredge• 6 months ago

Unlike the current American administration who condones raids on homes without warrants and justifies violence with lies, this France raid follows something called rule of law.

So no, don't be coy and pretend that all governments are like American institutions.

SilverBirch• 6 months ago

It's also just very basic police work. We're investigating this company, we think they've committed a crime. Ok, why do you think that. Well they've very publicly and obviously committed a crime. Ok, are you going to prosecute them? Probably. Have you gone to their offices and gathered evidence? No thanks.

Of course they're going to raid their offices! They're investigating a crime! It would be quite literally insane if they tried to prosecute them for a crime and how up to court having not even attempted basic steps to gather evidence!

NooneAtAll3• 6 months ago

that's kinda the normalization argument, not the reason behind it

"it is done because it's always done so"

throwaway2902• 6 months ago

EU wants to circumvent e2e to fight CSA: "nooo think about my privacy, what happened to just normal police work?"

Police raids offices literally investigating CSA: "nooo police should not physically invade, what happened to good old electronic surveillance?"

londons_explore• 6 months ago

A company I worked for had a 'when the police raid the office' policy, which was to require they smash down the first door, but then open all other doors for them.

That was so that later in court it could be demonstrated the data hadn't been handed over voluntarily.

They also disconnected and blocked all overseas VPN's in the process, so local law enforcement only would get access to local data.

hybrid_study• 6 months ago

The people who think raids are pointless probably use TELNET instead of SSH :-)

almosthere4• 6 months ago

At some point in the near future I see a day where our work laptops are nothing more than a full screen streaming video to a different computer that is housed in a country that has no data extradition treaties and is business friendly.

Because that country and the businesses that support that are going to get RICH from such a service.

neoromantique2• 6 months ago

In some places that day is today, and has been for a while.

AtlasBarfed• 6 months ago

Sealand,!!!!!!

dragonwriter4• 6 months ago

At some point in the near future I see a day where our work laptops are nothing more than a full screen streaming video to a different computer that is housed in a country that has no data extradition treaties and is business friendly.

Do you mean they will be pure worker surveillance systems, or did you mean “from” instead of “to”?

vpark• 6 months ago

Not video but that's essentially how companies like Google operate today. That's why their engineers can use an off the shelf Chromebook. Their IDE is on the web, etc.

TitaRusell2• 6 months ago

Did you miss what happened to Maduro?

utopiah4• 6 months ago

Takes literally minutes to setup with Webtop (assuming you are familiar with Docker/Podman https://docs.linuxserver.io/images/docker-webtop/ ) , nothing to install on the thin client, the stock browser is enough.

I used this when an employer was forcing me to use Windows and I needed Linux tools to work efficiently so I connected home. Goes through firewalls, proxies, etc.

Anyway if you want to host this not at home but a cloud provider there was HavenCo https://en.wikipedia.org/wiki/HavenCo don't ask me how I know about it, just curiosity.

SoftTalker• 6 months ago

They don't just take paper when they raid offices. They take the computers too. I've never worked anywhere where desktop machines are encrypted as a matter of routine. Laptops yes (but only recently). Servers maybe, depending on what they do.

Foobar8568• 6 months ago

Encryption won't save you from warrants and judiciary processing anywhere in the world.

notepad0x90• 6 months ago

Yeah, two things to add:

1) Even when you move things to a server, or remove it from your device, evidence is still left over without your knowledge sometimes.

2) Evidence of data destruction, is in itself as the name implies, evidence. And it can be used to prove things.

For example, an ext4 journal or NTFS USN $J journal entry that shows "grok_version_2.4_schema.json" where twitter is claiming grok version 2.4 was never deployed in France/UK is important. That's why tools like shred and SDelete rename files before destroying them. But even then, when those tools rename and destroy files, it stands out, it might even be worse because investigators can speculate more. It might corroborate some other piece of evidence (e.g.: sdelete's prefetch entry on windows, or download history from a browser for the same tool), and that might be a more serious charge (obstruction of justice in the US).

utopiah4• 6 months ago

Indeed, actions leaves traces, including the action of deleting data. It takes a LOT of expertise to be able to delete something without leaving a trail behind, if that's even feasible without going to extraordinary length.

Animats• 6 months ago

One of the charges is "fraudulent data extraction by an organised group." That's going to affect the entire social media industry if applied broadly.

muyuu5• 6 months ago

Frankly it sounds to me like a "show me the man and I'll show you the crime" kind of operation. France and the UK, and judging by yesterday's speech by the PM of Spain maybe the whole EU might be looking to do what China and Russia did earlier on and start cracking down on foreign social media by making it impossible to operate without total alignment with their vision and not just their (new) rules. Together with a push for local alternatives, that currently don't seem to be there, it may spell the end for a big chunk of the Global social network landscape.

I still believe that the EU and aligned countries would rather have America to agree to much tighter speech controls, digital ID, ToS-based speech codes as apparently US Democrats partly or totally agree to. But if they have workable alternatives they will deal with them from a different position.

Palmik• 6 months ago

EU might be looking to do what China and Russia did earlier on and start cracking down on foreign social media

For some reason you forgot to mention "Like the US did with TikTok".

PaulRobinson• 6 months ago

It's worth pointing out that in France and the UK, the authorities involved are arms length independent of the political bodies - it's not like the US where if you give the President good vibes you can become head of the FBI, and all you have to do in return is whatever he says. There are statutory instruments (in France, constitutional clauses), that determine the independence of these authorities.

They are tasked - and held to account by respective legislative bodies - with implementing the law as written.

Nobody wrote a law saying "Go after Grok". There is however a law in most countries about the creation and dissemination of CSAM material and non-consensual pornography. Some of that law is relatively new (the UK only introduced some of these laws in recent years), but they all predate the current wave of AI investment.

Founders, boards of directors and their internal and external advisors could:

1. Read the law and make sure any tools they build comply

2. When told their tools don't comply take immediate and decisive action to change the tools

3. Work with law enforcement to apply the law as written

Those companies, if they find this too burdensome, have the choice of not operating in that market. By operating in that market, they both implicitly agree to the law, and are required to explicitly abide by it.

They can't then complain that the law is unfair (it's not), that it's being politicised (How? By whom? Show your working), and that this is all impossible in their home market where they are literally offering presents to the personal enrichment of the President on bended knee while he demands that ownership structures of foreign social media companies like TikTok are changed to meet the agenda of himself and his administration.

So, would the EU like more tighter speech controls? Yes, they'd like implementation of the controls on free speech enshrined in legislation created by democratically appointed representatives. The alternative - algorithms that create abusive content, of women and children in particular - are not wanted by the people of the UK, the EU, or most of the rest of the World, laws are written to that effect, and are then enforced by the authorities tasked with that enforcement.

This isn't "anti-democratic", it's literally democracy in action standing up to technocratic feudalism that is an Ayn Randian-wet dream being played out by some morons who got lucky.

yxhuvud2• 6 months ago

Yes, if you don't follow EU laws prepare to not do business in Europe. Likewise, if you don't follow US laws I'd advise against trying to do business in USA.

mnewme7• 6 months ago

Good one.

No platform ever should allow CSAM content.

And the fact that they didn’t even care and haven’t want to spend money for implementing guardrails or moderation is deeply concerning.

This has imho nothing to do with model censorship, but everything with allowing that kind of content on a platform

ReptileMan6• 6 months ago

I disagree. Prosecute people that use the tools, not the tool makers if AI generated content is breaking the law.

A provider should have no responsibility how the tools are used. It is on users. This is a can of worms that should stay closed, because we all lose freedoms just because of couple of bad actors. AI and tool main job is to obey. We are hurling at "I'm sorry, Dave. I'm afraid I can't do that" future with breakneck speed.

mnewme7• 6 months ago

I agree that users who break the law must be prosecuted. But that doesn’t remove responsibility from tool providers when harm is predictable, scalable, and preventable by design.

We already apply this logic elsewhere. Car makers must include seatbelts. Pharma companies must ensure safety. Platforms must moderate illegal content. Responsibility is shared when the risk is systemic.

thrance4• 6 months ago

How? X is hostile to any party attempting to bring justice to its users that are breaking the law. This is a last recourse, after X and its owner stated plainly that they don't see anything wrong with generating CSAM or pornographic images of non-consenting people, and that they won't do anything about it.

moorebob3• 6 months ago

But how would we bring down our political boogieman Elon Musk if we take that approach?

Everything I read from X's competitors in the media tells me to hate X, and hate Elon.

If we prosecute people not tools, how are we going to stop X from hurting the commercial interests of our favourite establishment politicians and legacy media?

kakacik• 6 months ago

You won't find much agreement with your opinion amongst most people. No matter of many "this should and this shouldn't" is written into text by single individual, thats not how morals work.

intended• 6 months ago

If you had argued that it’s impossible to track what is made on local models, and we can no longer maintain hashes of known CP, it would have been a fair statement of current reality.

——-

You’ve said that whatever is behind door number 1 is unacceptable.

Behind door number 2, “holding tool users responsible”, is tracking every item generated via AI, and being able to hold those users responsible.

If you don’t like door number 2, we have door number 3 - which is letting things be.

For any member of society, opening door 3 is straight out because the status quo is worse than reality before AI.

If you reject door 1 though, you are left with tech monitoring. Which will be challenged because of its invasive nature.

Holding Platforms responsible is about the only option that works, at least until platforms tell people they can’t do it.

tw85• 6 months ago

It seems people have a rather short memory when it comes to twitter. When it was still run by Jack Dorsey, CP was abundant on twitter and there was little effort to tamp down on it. After Musk bought the platform, he and Dorsey had a public argument in which Dorsey denied the scale of the problem or that old twitter was aware of it and had shown indifference. But Musk actually did take tangible steps to clean it up and many accounts were banned. It's curious that there wasn't nearly the same level of outrage from the morally righteous HN crowd towards Mr. Dorsey back then as there is in this thread.

misnome• 6 months ago

When did Jack Dorsey unban personal friends of his that had gotten banned for posting CSAM?

brightball3• 6 months ago

I meant to reply to you with this: https://news.ycombinator.com/item?id=46886801

ryandrake• 6 months ago

Didn't Reddit have the same problem until they got negative publicity and were basically forced to clean it up? What is with these big tech companies and CP?

AaronFriel• 6 months ago

Didn't X unban users like Dom Lucre who posted CSAM because of their political affiliation?

ceejayoz• 6 months ago

But Musk actually did take tangible steps to clean it up and many accounts were banned.

Mmkay.

https://en.wikipedia.org/wiki/Twitter_under_Elon_Musk#Child_...

"As of June 2023, an investigation by the Stanford Internet Observatory at Stanford University reported "a lapse in basic enforcement" against child porn by Twitter within "recent months". The number of staff on Twitter's trust and safety teams were reduced, for example, leaving one full-time staffer to handle all child sexual abuse material in the Asia-Pacific region in November 2022."

"In 2024, the company unsuccessfully attempted to avoid the imposition of fines in Australia regarding the government's inquiries about child safety enforcement; X Corp reportedly said they had no obligation to respond to the inquiries since they were addressed to "Twitter Inc", which X Corp argued had "ceased to exist"."

jmcgough• 6 months ago

Having an issue with users uploading CSAM (a problem for every platform) is very different from giving them a tool to quickly and easily generate CSAM, with apparently little-to-no effort to prevent this from happening.

brightball3• 6 months ago

Agreed. For anyone curious, here's the UK report from the National Society for the Prevention of Cruelty to Children (NSPCC) from 2023-2024.

https://www.bbc.com/news/articles/cze3p1j710ko

Reports on sextortion, self-generated indecent images, and grooming via social media/messaging apps:

Snapchat 54%

Instagram 11%

Facebook 7%

WhatsApp 6-9%

X 1-2%

rcpt• 6 months ago

What are the percentages?

jbenner-radham• 6 months ago

Are those numbers in the article somewhere? From what I read it says that out of 7,062 cases, the platform was known for only 1,824. Then it says Snapchat accounts for 48% (not 54%). I don't see any other percentages.

yibg2• 6 months ago

My natural reaction here is like I think most others; that yes Grok / X bad, shouldn't be able to generate CSAM content / deepfakes.

But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user. I've seen a few arguments here that mostly comes down to:

1. It's Grok the LLM generating the content, not the user.

2. The distribution. That this isn't just on the user's computer but instead posted on X.

For 1. it seems to breakdown if we look more broadly at how LLMs are used. e.g. as a coding agent. We're basically starting to treat LLMs as a higher level framework now. We don't hold vendors of programming languages or frameworks responsible if someone uses them to create CSAM. Yes LLM generated the content, but the user still provided the instructions to do so.

For 2. if Grok instead generated the content for download would the liability go away? What if Grok generated the content to be downloaded only and then the user uploaded manually to X? If in this case Grok isn't liable then why does the automatic posting (from the user's instructions) make it different? If it is, then it's not about the distribution anymore.

There are some comparisons to photoshop, that if i created a deep fake with photoshop that I'm liable not Adobe. If photoshop had a "upload to X" button, and I created CSM using photoshop and hit the button to upload to X directly, is now Adobe now liable?

What am I missing?

realusername2• 6 months ago

But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user.

Because Grok and X aren't even doing the most basic filtering they could do to pretend to filter out CSAM.

flumpcakes• 6 months ago

For 1. it seems to breakdown if we look more broadly at how LLMs are used. e.g. as a coding agent. We're basically starting to treat LLMs as a higher level framework now. We don't hold vendors of programming languages or frameworks responsible if someone uses them to create CSAM. Yes LLM generated the content, but the user still provided the instructions to do so.

LLMs are completely different to programming languages or even Photoshop.

You can't type a sentence and within 10 seconds get images of CSAM with Photoshop. LLMs are also built on trained material, unlike the traditional tools in Photoshop. There have been plenty CSAM found in the training data sets, but shock-horror apparently not enough information to know "where it came from". There's a non-zero chance that this CSAM Grok is vomiting out is based on "real" CSAM of people being abused.

dragonwriter4• 6 months ago

But I am having trouble justifying in an consistent manner why Grok / X should be liable here instead of the user.

This seems to rest on false assumptions that: (1) legal liability is exclusive, and (2) investigation of X is not important both to X’s liability and to pursuing the users, to the extent that they would also be subject to liability.

X/xAI may be liable for any or all of the following reasons:

* xAI generated virtual child pornography with the likenesses of actual children, which is generally illegal, even if that service was procured by a third party.

* X and xAI distributed virtual child pornography with the likenesses of actual children, which is generally illegal, irrespective of who generated and supplied them.

* To the extent that liability for either of the first two bullet points would be eliminated or mitigated by absence of knowledge at the time of the prohibited content and prompt action when the actor became aware, X often punished users for the prompts proucing the virtual child pornography without taking prompt action to remove the xAI-generated virtual child pornography resulting from the prompt, demonstrating knowledge and intent.

* When the epidemic of grok-generated nonconsensual, including child, pornography drew attention, X and xAI responded by attempting to monetize the capacity by limiting the tool to only paid X subscribers, showing an attempt to commercially profit from it, which is, again, generally illegal.

lux-lux-lux• 6 months ago

The lack of guardrails wasn’t a carelessness issue - Grok has many restrictions and Elon regularly manipulates the answers it gives to suit his political preferences - but rather one of several decisions to offer largely unrestricted AI adult content generation as a unique selling point. See also, e.g. the lack of real age verification on Ani’s NSFW capabilities.

dominicrose• 6 months ago

A Russian in a French prison says my country isn't free. Well, let that message spread to other criminals. You're not welcome in France.

tokai4• 6 months ago

Good old HN. Users losing their collective minds over rule of law and CSAM being bad.

y-curious• 6 months ago

Found the guy who supports digital ID and involuntary phone database scanning

tokai4• 6 months ago

Found the cheese pizza enjoyer.

ddtaylor5• 6 months ago

HN is capable of a better discussion without personal attacks.

code_for_monkey3• 6 months ago

extremely cursed userbase sometimes

hearsathought2• 6 months ago

CSAM being bad.

Agreed. It's why it's difficult to support France who has sheltered Roman Polanski for decades.

It's strange how people like you only think it's bad when it suits your political agenda.

krautburglar• 6 months ago

Raid all of them. Raid Google. Raid Facebook. Raid Apple. Raid Microsoft. Big tech has gotten away with everything from fraud[0] to murder[1] for decades. Black outfits. Rappel lines. Automatics. Touch that server Prakesh, and you won't live to touch another.

[0] https://nypost.com/2025/12/15/business/facebook-most-cited-i... [1] https://en.wikipedia.org/wiki/Suchir_Balaji

patrick451• 6 months ago

Until the EU gets their act together on free speech, it's high time the united states hits them with aggressive economic sanctions.

poseva• 6 months ago

Is this sarcasm ? because sounds like sarcasm. If Grok generating naked images of children is your ideea of free speech no wonder nobody takes a stand against Trump and Co., you somehow managed turn your common sense to dust...

hearsathought2• 6 months ago

Europe/EU should have whatever free speech they want. But we should annex all european controlled territory in the western hemisphere. The amount of territory that european empires still control in this hemisphere is alarming and frankly embarrassing. From greenland to "french" guyana to the falklands, it should all be seized.

coffeebeqn• 6 months ago

https://ourworldindata.org/grapher/freedom-of-expression-ind...

drawfloat• 6 months ago

Settings aside this is about CSAM, the US is the only one of the two to shut down a foreign social network because it dislikes what was said on it. The US doesn't get to play that card anymore.

voidUpdate• 6 months ago

"the only country in the world that is criminally persecuting all social networks that give people some degree of freedom"

Sounds like he's never been to Russia. Which is weird, given that he's Russian

arielcostas• 6 months ago

Yeah, and in his mind the only social networks that exist are Telegram and Twitter. Mastodon doesn't exist, bluesky doesn't either, rr any of Meta's products (including Threads, direct competitor to Twitter with seemingly more bots and propaganda)

fer• 6 months ago

Yeah, as someone who's been to Russia pre-war, and has worked and befriended Russians, the beam in the eye is quite strong in many. It's striking because the social contract there is basically that you're left alone as long as you don't become a thorn of the power, social network-mediated or not, and I'm not even talking about Putin and his entourage, but local administration in a remote village too. Zvyagintsev portrays it royally in Leviathan, as it's been portrayed by Dostoyevsky and Tosltoy before.

My most recent case: I went on holiday to a resort in Turkey, numerous Russians, families, retired, etc. I don't pass as a Russian-speaker (but I understand quite well) and once they hear me talking other unrelated language they naturally start to speak more freely in front of me (i.e. more liberal use of swearing, and even slurs if no other Russians are around).

While sunbathing, or at the restaurant, or the pool, they were talking about daily, mundane things, same in the restaurant, etc. But when floating in pairs 20-30m from the shore? Politics.

code_for_monkey3• 6 months ago

lord some of the people on this site need their hard drives searched.

ddtaylor5• 6 months ago

I was once closer to s free speech absolutist, but this has always been where that discussion leads. The lines were clearer decades ago because someone who interacted with CSAM was closely involved and their harm was easier to see in totality. Over time the gap between people who caused that harm and those that think they are not hurting anyone has widened. With AI it caused many of those that thought it was acceptable to consume illegal content to put another layer between their victims and offload the moral baggage.

throwaway2902• 6 months ago

To me it became clear when people here rejected any compromises on privacy and anonymous on device scanning of messages/data. Not because they rejected it but how they rejected it.

People here didn't say "yeah it's a real evil problem and e2e tech that we build makes it almost impossible to catch and helps it scale, so let's find the right tradeoff that minimizes privacy invasion". No. People here just mocked "think of the children" and said something along the lines of no amount of suffering can make any restriction to privacy be acceptable. The fact that 99% of our life is literally compromising our freedom for others they don't care.

To buellerbueller's killed comment, I totally feel it. it was a shocker when it turned out many of my fellow Russians got no problem with Putin's war and I bet that's what half of USA feels.

wnevets• 6 months ago

The man that is friends with a bunch of pedophiles owns a website that is becoming known for generating CSAM? What are the chances?

devwastaken• 6 months ago

E.U is replacing their citizens with unvetted violent criminals. They have to vote for whoever gives them free stuff. The powers doing this are upset a company in the U.S allows their citizens to protest it.

mekdoonggi5• 6 months ago

Read that on X?

Pedro_Ribeiro3• 6 months ago

Do you honestly not believe it is likely that EU is importing votes?

Also, haven't you seen the general push towards censorship, attempts to ban VPNs, and all the other shenanigans happening in the EU? Do you believe this is disconnected from the legal attempts on Twitter and Telegram?

Is it really a conspiracy theory at this point? Politicians do all kinds of evil shit, but these playbook tactics are where you draw the line?

I'm European and live here, before you say I'm getting these takes on X.

Havoc• 6 months ago

The lack of legal/regulatory reaction US side on CSAM is pretty bewildering. And ominously mirrors the Epstein saga - other countries seem to be moving fast while US seems to be trying very hard to not move at all

X offices raided in France as UK opens fresh investigation into Grok

Related Stories