One Dataset. No Warning. Google Took Everything. You're Not Safe Either

https://medium.com/@russoatlarge_93541/i-built-a-privacy-app-google-banned-me-over-a-dataset-used-in-ai-research-66bc0dfb2310

by markatlarge • 12 months ago

111 68 12 months ago

medium.com

sidkris• 12 months ago

Have you considered creating a PWA?

nerdsniper2• 12 months ago

I encrypt so much of the stuff I upload to google drive or email. In my case, more because my filetypes and contents tend to trigger malware detection (even though literally none of it is malware or even security research).

Stories like this are why I do it. I don’t know when something is going to get flagged - NSFW or otherwise. I really should de-google and I mean to. Buts its a daunting project for the email address I’ve had since 2004 (my email is now old enough to drink in USA).

beeflet2• 12 months ago

I was in the same position. here's what I did:

1) Got my own domain

2) Subscribed to an email service that lets you use your own domain (for example fastmail)

3) Forwarded all of my email from my gmail account to my new email/domain, and use my new email/domain in all correspondence.

4) Made a separate google account for every google service I used. For example, I made a separate account from my gmail for google play, google cloud and youtube.

It's a bit of work but this allowed me to slowly ease myself out of gmail, and derisk my account activity. Even if fastmail screws up, I can always point my domain at another email provider like protonmail.

Oh, also:

5) Use syncthing for file storage. It's cheap and I can back up TB worth of stuff from decades ago.

physicles• 12 months ago

I want to do this too, but what’s your threat model around losing access to your domain name?

To maintain a Gmail account, you need to:

1. Do nothing

2. Don’t be unlucky

To maintain a domain name, you need to:

1. Keep a functioning credit card forever

2. Make sure the domain name is renewed forever

3. Take action if your registrar goes out of business

The odds of something going wrong with your domain name, while still low, seem higher. Are you banking on the fact that you can talk to a human at your registrar to resolve issues?

nerdsniper2• 12 months ago

(4) doesn’t necessarily help due to googles “Sensorvault”. I’ve read stories where one employee's personal account ban (or ex-employee) resulted in all “related accounts” getting banned as well, locking out an entire business due to some perceived violation on someones personal account, and other employee's personal accounts got suspended by association as well: https://www.reddit.com/r/tifu/comments/8kvias/tifu_by_gettin...

https://old.reddit.com/r/androiddev/comments/ts6jfg/google_h...

Also using multiple accounts can result in suspension: https://news.ycombinator.com/item?id=34961069

selfhoster112• 12 months ago

At the very least, do a Google Takeout. One step forward.

isaacremuant• 12 months ago

I know. They've been doing this for ages. It's very hard to convince people something like this can happen to them until it does.

Fade_Dance4• 12 months ago

Currently going through this with Reddit. Their appeal process is to submit a form which may or may not be processed, and there is zero feedback for the user. Common advice is to submit the form daily.

I have submitted it 200 times over the past year. I know that it probably just goes into a black void, but it is apparently the official review process, and I am 100% sure there were no rule violations (I was flagged for responding to new threads too quickly and reposting the same educational link a few times because I was answering the same sort of question). It's cruel really. Somehow they made something even worse than the Google approach. Presumably tens of thousands of users are wasting time every single day doing this.

Like others have said, most tech companies are like this and it's unacceptable, even if users are the product.

ryandrake3• 12 months ago

Why would you keep trying 200 times to use a site that hates and ignores you? Just find something else to do online.

If HN suddenly banned my account out of nowhere for no discernible reason, I’m not going to E-mail dang 200 times. I’d just find some other site to read. Life is too short to do business with companies that despise you.

Fade_Dance4• 12 months ago

Well I have a text file with 5 boiler plate messages. When I boot the screens up every day, I open a new tab and 7 seconds later a new appeal has been submitted.

I do admit that there is a slight prick of annoyance at the start of every day because of this, but I'm well into sunk cost fallacy into the realm of morbid curiosity and flagellation.

Part of the reason is because I manage an account that has a major position in Reddit equity options, so I have to read the name everyday regardless. Indeed I was banned because my option trading advice was too quick and frequent (man, that sub needs some serious basic theory help)... Of course, you can be assured that their horrible customer goodwill may possibly come up in future conversations when I mention the position.

I thought about just emailing IR, but I've actually talked to someone else in a similar position who has contacted IR and they did nothing either.

I was a major contributor to an open source Reddit alternative after the API lockdown, but unfortunately after 2 years I just didn't think that it was going anywhere big and backed out of that, so been there done that to some degree, and that was a good 500 plus hours of investment (although it was net positive - it was fun to grow an old school internet community even though it is never going to compete with Reddit).

The main issue is that they ban all accounts that they can fingerprint to other banned accounts, and I have an account in hibernation that is 15 years old and truly has a huge portion of my life contained in it. It's much like OP and losing a Google account. I think that is the main reason why I grovel to the uncaring corporate shell that encompasses the community that I once help to build when I came over in the great digg migration. I should probably just export the data and vectorize it into an LLM, and just close the book on the few old friends that I'd like to reconnect with someday.

Speaking of Digg, I hear they're rebooting, and things are going well behind the scenes...

markh1967• 12 months ago

Why go to this much trouble for a Reddit account?

A few months ago I browsed r/all by new and downvoted every crypto scam post to unrelated subreddits I saw. This raised a red flag somewhere inside Reddit (my guess is that I was interfering with some Reddit admin's crypto scams) and my 18 year old account was suspended until I could respond to an email at the address it was registered from that I no longer had access to almost two decades later. An appeal was ignored.

I was furious about losing the account for about two minutes before I realised that nothing of any importance had been lost, shrugged, and created a new account.

more_corn4• 12 months ago

What’s so important about your Reddit account that you would file a request to restore 200 times? Also, no judgement, but that sounds a little weird.

jalapenos4• 12 months ago

Good public warning.

But still, "unzip random NSFW content onto Google Drive" doesn't sound risk free does it

josephcsible2• 12 months ago

It doesn't sound risk free, but in a reasonable world, the risk would just be accidentally downloading a virus or something and having to wipe your PC, not being unpersoned for life by Google.

jalapenos4• 12 months ago

I think at the very least, Google should offer a paid service where you can actually talk to someone who'll actually look into it.

Even if it's $1000 a case, people will often still pay it, and that'd nullify the "scalability of customer service" crap.

msgodel• 12 months ago

The name "Google drive" implies it's like a hardware disk: unaware of even the filesystem, much less the files, stored on it. Most users likely think of it that way and don't notice until they accidentally trigger things.

jalapenos4• 12 months ago

The word Google in that name implies that it is Google's

andrewinardeer• 12 months ago

Not surprised.

Google clearly states CSAM is forbidden on their platform.

The user stored CSAM on Google Drive. Regardless whether it is for academic research or not - which in this case, it wasn't. It was for training a model.

Do you really think Google wants to be on the front page of the newspaper saying it allows users to store CSAM?

The user has been caught off guard and that's on them.

And yes, you're not safe either if you store CSAM on Google. Who in their right mind thinks they are?

Edit: Grammar

like_any_other2• 12 months ago

CSAM is the author's baseless speculation, but I could find no evidence suggesting NudeNet contains any. That it's hosted on GitHub, all its files are on archive.org, and is a widely used data-set 6 years old by now, all indicate that it doesn't contain CSAM.

In general, when a company does something hostile and unreasonable, let's not invent out of thin air facts to excuse their behavior, shall we? Especially since their stonewalling is deliberate.

markatlarge4• 12 months ago

Google won’t tell me the name of the file—or files—they believe violated their policy. So IT IS baseless speculation. I only know what I was doing prior to the account deletion—and I’m guessing that’s what triggered the issue.

Google should at minimum publish the filenames of the content they consider violations. I have Colab notebooks for everything I’ve downloaded from the internet. With a filename, I could trace the origin and have the image independently reviewed.

I suspect it wasn’t CSAM at all. We know nothing about how their detection system or hashing works. I’d also like to point out that I benchmarked my model against Google’s own commercially available NSFW detector—and mine actually performed better: https://medium.com/@russoatlarge/benchmark-study-punge-yolov...

As for datasets: COCO is the most widely used dataset in computer vision, with hundreds of thousands of labeled images. One image in the COCO Train2017 set—000000001790.jpg—shows a child brushing his teeth while looking in the mirror in his “birthday suit.” But if Google's system flagged that as CSAM, then every researcher using COCO is at risk.

If Google genuinely cared about addressing CSAM responsibly, they would share the filenames so they could be independently verified and—if needed—removed from circulation. Instead, they’re silent. That silence creates fear, not safety.

markatlarge4• 12 months ago

You’re making a serious accusation without evidence. I don’t know if the file in question was CSAM—because Google won’t tell me. We just have Google’s word for it.

What I do know is that I used publicly available datasets—like NudeNet and COCO—for benchmarking an NSFW detection model. These datasets have been used by researchers for years. If there was anything questionable in them, that should be addressed openly—not used as a reason to erase someone’s digital life without transparency or due process.

I’m not claiming academic immunity. I’m saying it was unintentional, and I had no idea anything I processed could have been flagged. And if there was something problematic in a dataset, I would absolutely want to identify and report it—but I can’t do that if Google refuses to disclose what was flagged.

This isn’t just about me. When a private company has the unchecked power to destroy someone’s livelihood overnight, it stops being a simple platform—it becomes a de facto utility. There needs to be a path for review, correction, and accountability.

More on what happened to me here: https://medium.com/@russoatlarge/when-youre-accused-youre-er...

afandian2• 12 months ago

I got into a situation a couple of years ago with Google over a few dollars when I terminated my paid GSuite account. It was impossible to contact them to settle the balance. They kept sending emails from a Google-branded department that was clearly outsourced. The process was literally Kafka-esque.

Didn’t manage it in the end. Ended up going to collections who happily listened and cancelled the debt.

I was a happy paying email customer of many years.

I now am sworn never to enter any kind of relationship with Google, and encourage to do the same.

Mo3• 12 months ago

I still wonder why they literally don't seem to care in Europe. I've overdrawn billing accounts (by accident) two times, one 10 years ago, one a year ago. Nothing happened until I remembered about them months and months later, I went to check and both times the accounts were just sitting there closed. Checked my email and they sent like two payment reminders in the first 2 weeks and then nothing ever again.. the one 10 years ago is still sitting there.. it's not like they couldn't have sent it to debt collections, that's also a popular thing here lol.

afandian2• 12 months ago

That’s legacy thinking. Customers are cattle not pets!

I’m in the UK btw.

immibis4• 12 months ago

If it was Germany I expect them to find a way to legally upgrade the debt to $99,999 or so, and then take you to court, and the court will side with them and seize your assets to pay it. German law doesn't fuck around; unfortunately the precise ways in which it doesn't fuck around are arcane and only accessible to billionaires. And if you get it wrong the entire cost backfires on you.

more_corn4• 12 months ago

Same. They did a “funny” thing where they routed my balance owed refund request around the world twice I think I have 12 bounces in the thread. Every bounce was the same useless canned information.

For every hour I wasted on it I filed a complaint with a regulatory body. It probably cost the millions of dollars. I recommend everyone in a similar situation do the same.

nicomt• 12 months ago

Google suspended my payments account a few months ago without even notifying me. I never received a reason for the suspension, but I suspect it's related to a failed game refund from Stadia, as I see a refund error in my payment notifications. Dealing with Google Support has been a Kafkaesque loop where I have to explain the same thing over and over, only to get the same scripted instructions that take me nowhere. After this dance, they finally say they will "escalate" the issue and then close the case. At this point, I've given up and am in the process of de-googling my life.

crossroadsguy• 12 months ago

I was lucky enough to have had my (first - pretty nice little common username) Gmail account summarily culled towards the early years of my email usage. I was young and I hadn't used the email anywhere where it could get critical - I don't think I had anything critical in life at that time. Besides, my country hadn't moved to emails for personal communications and interactions - it never did, it just teleported into instant messaging.

I was never given a reason. I tried to reach out and, of course, we know how it ends (even then). The thing is, I didn't even use it for anything other than sending emails to friends who were starting in other colleges and kind of just using emails for the sake of using emails. It was the new tool for me. Except one - the teenage me was in long and cosy exchanges with a nice someone many states and thousands of kilometres over talking about Jane Austen and Tolstoy and their works that we both read and loved and what not and also sharing with each other things I now definitely would not call poetry. That still feels like a loss - even after some two odd decades. The silliest thing - I didn't remember the email. Those were the good old days of r1d1clulux_paynthr_56789_b0mb@rediffmail.com. We never asked or revealed each other's names. Those were days like that when you didn't have to worry about whether you were talking to a cat or a fish.

I started using other email services (Yahoo, Hotmail - for a long time those were my main emails) and when I started working I got a domain and moved to a paid email host.

This also taught me a lesson about using free services. Hell, even when using a paid service this can happen. Apple is an example. You feel Google does it, oh boy oh boy, Apple knows how to do the true stonewalling! When you are on the phone with a real human being and that human being stonewalls you and their company (Apple) doesn't even allow that human being to help you, or escalate, or move up in the support chain, you feel that is real something - esp. if you are in a country where consumer laws are less than jokes! Anyway, so that taught me to be ready, to have backups (not just as in data, but as in options), and fallbacks.

I think everyone should experience things like this, but kinda early in life :)

PS. Naah, OP's case isn't like mine. But then people like me (and maybe OP and maybe you) allowed and enabled (still do) Googles and Apples of this world to be this big that they can just do it and get away with it and it doesn't even matter even in countries where consumer laws have real functioning teeth. So that's there.

like_any_other2• 12 months ago

I fully support systems that prevent harmful content.

"'I never thought leopards would eat MY face,' sobs woman who voted for the Leopards Eating People's Faces Party."

Gigachad• 12 months ago

It's a ChatGPT generated article. Probably wouldn't bother reading too much in to it

selfhoster112• 12 months ago

ChatGPT generated or not, I had a situation like this too with another large platform. It's an issue I would like to see discussed more, whether this article's case is fake or not.

jachriga• 12 months ago

Doesn't matter how they wrote it, they put their name on it so it should be treated as if they wrote it themselves.

markatlarge4• 12 months ago

I'm a real person — this actually happened to me last Thursday.

My name is Mark Russo. I’m not going to be shamed or erased. I created an app called Punge that’s live on both iOS and Android. The article wasn't written by ChatGPT — it's my story. I used ChatGPT to help tighten the writing, but every word reflects what I lived through.

Here’s the app: iOS: https://apps.apple.com/us/app/punge/id6450322655?platform=ip... Android: https://play.google.com/store/apps/details?id=com.markatlarg... TikTok demo: https://www.tiktok.com/t/ZPHpvJJLHsQyK-Cfk4Y/

Since this happened, I’ve reached out to lawyers, media, lawmakers, and professional contacts — even people I know at Google. I’ve lost access to everything tied to my account: Gmail, Firebase, AdMob, and the infrastructure that powers my apps. Google won’t even tell me which file triggered the CSAM flag.

Even if — by some miracle — I get my account back, I’m committed to making sure this kind of thing doesn’t keep happening. No one should lose their digital life overnight, with no transparency, no due process, and no way to defend themselves.

According to this NYT article: https://www.nytimes.com/2022/12/30/technology/google-appeals... — people who manage to claw their way out of this nightmare usually do so only after getting media attention. A CBS reporter has shown some interest. If anyone here has been through something similar, message me on LinkedIn — I’ll do whatever I can to help share your story too.

Connect with me: LinkedIn: https://www.linkedin.com/in/mark-russo-60a46912/

markatlarge4• 12 months ago

We agree I fully support systems that protect children,

Read my medium article

"When You’re Accused, You’re Erased" https://medium.com/@russoatlarge_93541/7a09ebe8bf6a In the article I say:

"A Necessary Fight — But Accountability Matters

Let me be clear: I deeply appreciate Google’s efforts to detect and report CSAM. These systems are vital for protecting children and stopping abuse, and I fully support that mission.

But good intentions are not a substitute for fairness. Right now, individuals who are falsely flagged have no way to defend themselves, no way to clear their name, and no meaningful path to restore their livelihood.

If a system is powerful enough to destroy someone’s life, it should also be strong enough to offer transparency, review, and correction. That’s all I — and others like me — are asking for: a fair process to protect the innocent while continuing the fight against real harm."

TheBozzCL• 12 months ago

If you have anything even remotely important tied to your email address… then at least get your own domain name associated with it. It’s not a 100% guarantee that things won’t go wrong, but at least you can recover if your current digital feudal lord turns their ire on you.

RagnarD• 12 months ago

All evidence to date forces the conclusion that trusting Google for anything important is tremendously foolish. That includes email, cloud-hosting, search, google apps of any kind, etc.

andrewstuart• 12 months ago

“Trusted cloud”

“Lost everything”

“I made no backups”

It’s becoming a common lament.

Folks, please note you need backups of data if losing it would be a problem.

That is all.

josephcsible2• 12 months ago

This wasn't just data loss and wouldn't have been prevented by having backups. In particular, note that he can't receive new emails or publish updates to his apps anymore. And even if it were just data loss, it's still entirely Google's fault, and victim blaming to suggest otherwise.

snvzz• 12 months ago

They probably paid google for their cloud backup services.

raxxorraxor4• 12 months ago

That and less shitty hosters like Google. There are countless alternatives that don't put private data under any scrutiny.

rekabis2• 12 months ago

All professional and personal eMail through gMail?

Oof, buddy. That cock-up is on you. Even if you don’t want to administer your own iron, there are plenty of third-party services that can host your eMail domain.

And with a little work, there is eMail-in-a-box for a turnkey solution, and others like hMailServer if you just want a minimalistic low-effort server running on an old Windows box in your basement.

harshalizee• 12 months ago

No it's not. The absolute vast majority of professional and personal emails go through the big 3, Gmail, Yahoo, msn/hotmail/outlook. What we need is better laws to ensure email is treated similar to utilities with strong consumer protection and appeal system, not victim blaming.

raxxorraxor4• 12 months ago

I think this is vastly different in countries outside the US, where self hosted mail servers are still very common.

rekabis2• 11 months ago

I’m talking about not putting all your eggs into one basket. Of which OP’s nightmare is the perfect example.

I wouldn’t lean on a Gmail eMail when building Android apps in the same way I wouldn’t lean on an iCloud eMail when building iOS apps. If they get ticked off at you, they can cause you to lose everything. They can pull the rug out from under you at a moment’s notice. And while I have accounts under both, they’re throwaways that only see use as backup eMail.

It’s why you always need to fully separate your concerns. Use a third-party hosting that won’t otherwise skullf*k you because of something you have done elsewhere or with a service they don’t own. I mean, if they don’t own that service, why would they shaft you?

Havoc• 12 months ago

Sticking what is functionally a mountain of porn onto Google cloud drive…wow

Unfortunately that it got csam flagged and rubbish support options is classic Google but still a pretty bold move on authors part

raxxorraxor4• 12 months ago

This is why you either are a good cloud provider and don't put your users under surveillance or you have a Google cloud provider.

Although this is to a large degree a legislative issue as well. The general idiots in politics problem.

benreesman• 12 months ago

Should have remembered to use Incognito mode, everyone knows that you can't use your main browser for ... research.

efilife• 12 months ago

Why did people suddenly start to call CP CSAM? Did I miss something?

immibis4• 12 months ago

The stated reason is that because there's nothing wrong with pornography, using the term "pornography" makes it sound like there's nothing wrong with it.

The real reason seems to be the opposite - to make it sound less bad by making it an unfamiliar term. Like if instead of saying "the murderer" someone would say "the root cause if the cessation of cellular metabolic activity".

more_corn4• 12 months ago

Don’t download porn datasets to Google drive.

One Dataset. No Warning. Google Took Everything. You're Not Safe Either

Related Stories