Your AI Girlfriend Is a Data-Harvesting Horror Show

https://gizmodo.com/your-ai-girlfriend-is-a-data-harvesting-horror-show-1851253284

by nickthegreek • 2 years ago

146 194 2 years ago

gizmodo.com

chrisjj• 2 years ago

Subject needs correction to: Your AI Girlfriend Is a Data-Harvesting Horror Show

codetrotter• 2 years ago

Yeah. This is probably because of HN automatic removal of some words from headlines to make them less clickbaity.

But in this case on first reading of the title I thought it was about a new TV show called “AI Girlfriend”. And then I thought or maybe they are referring to those apps that people can install to have an AI gf.

mat_epice• 2 years ago

Nine times out of ten it's an electric razor, but every once in a while, it's an AI Girlfriend. Of course it's company policy never to imply ownership in the event of an AI Girlfriend. Always use the indefinite article an AI Girlfriend, never your AI Girlfriend.

franky47• 2 years ago

First rule of AI girlfriends: don't talk about AI girlfriends.

Sohcahtoa82• 2 years ago

I don't own...

nickthegreek2• 2 years ago

Thanks, I was able to update.

loceng2• 2 years ago

Step 1) Have AI girlfriend direct you to show your naked body, what you're doing "to her" - capturing very high-resolution video and audio.

Step 2) Generate blackmail materials of you involved in sexual acts in very high resolution with minors-children.

Step 3) Extort OR "leak" into public domain as part of a smear campaign to attempt to either 3.1) disempower you or 3.2) manufacture consent for the ideological mob - or soldier who's been indoctrinated into a truthful looking narrative - to attack or assassinate you.

Integrate the shadow if you read the above and don't believe it's possible and will be tried.

We're coming into very interesting times.

dylan6046• 2 years ago

You have a twisted definition of interesting.

beaeglebeached• 2 years ago

This is why child abuse material prosecution should require testimony of a victim-accuser to face. As law currently stands it's as easy as the ol "we searched his car and lo and behold look at this crack rock" trick.

gherkinnn• 2 years ago

I found many Black Mirror episodes to be a bit much. And then, out of nowhere, that shit becomes reality in one form or another. Charlie Brooker was way ahead of me. He even got the PM fucking a pig right.

ToucanLoucan9• 2 years ago

So how many more rounds of this cycle do we need before we leverage the letter of the law to say that maybe companies shouldn't be allowed to blatantly exploit people's vulnerability and isolation to make money? Or is your vulnerability to exploitation still the justification for your exploitation in the tech space?

jayrot• 2 years ago

Two or three. Four max.

Maybe five.

blitzar• 2 years ago

Possibly six or seven.

Absolutely no way more than eight.

glitchc4• 2 years ago

I don't know if humanity can function without the ability for humans to exploit each other.

ToucanLoucan9• 2 years ago

Maybe we should try? You know, mix things up a little? Variety is the spice of life after all.

philip1209• 2 years ago

Hasn't happened so far with social media.

u32480932048• 2 years ago

But it helps with Spongebob rainbow hands MENTAL HEALTH

gadders3• 2 years ago

Does that include p0rn?

01HNNWZ0MV43FF5• 2 years ago

Yes privacy laws should definitely protect the identity of porn viewers. I have some very embarrassing fetishes, and everyone knows that half of IT workers are furries.

almatabata7• 2 years ago

It is a good question. Not exactly porn per say but what is the difference between a human onlyfans model selling human interaction and a company selling access to an AI model? In the end you are exploiting somebodies need for intimacy.

dredmorbius• 2 years ago

Rather.

Ashley Madison <https://en.wikipedia.org/wiki/Ashley_Madison_data_breach>

AdultFriendFinder <https://www.zdnet.com/article/adultfriendfinder-network-hack...>

Also Lucious, Brazzers, and others.

dylan6046• 2 years ago

Would this apply to psychology/psychiatry/therapy too?

charlie0• 2 years ago

That one online help company got hacked, so even assuming they aren't malicious, having all that personal psych data under one roof makes them a juicy target.

01HNNWZ0MV43FF5• 2 years ago

That'd be great. My therapist doesn't put her notes into an EMR or anything, so there's no data warehouse where my credit card, legal name, SSN, and personal nightmares all live in the same SQL row.

rsynnott5• 2 years ago

... I mean, certainly for psychiatry it _does_; your psychiatrist absolutely cannot sell your personal information to random marketing companies, and in general medical confidentiality is fairly strong. The others will vary by jurisdiction based on how or if they're regulated.

commandlinefan• 2 years ago

There's a reason intelligence agencies recruited pretty girls to seduce high-ranking officials during the cold war.

api5• 2 years ago

This practice goes back to the ancient world and I'm sure continues today.

Jeffrey Epstein seemed to be doing things like that, whether for his own benefit (financial fraud) or on behalf of others. Maybe both.

wombat-man• 2 years ago

I can't tell if he was a gov agent, or just extremely dedicated to sexual perversion.

bdw5204• 2 years ago

It is definitely still happening today. The most famous recent example would probably be Maria Butina[0], a Russian woman who infiltrated the National Rifle Association and the US Republican Party during the 2016 election cycle to influence US politics. She was arrested, convicted of federal crimes and eventually deported back to Russia where she is now a member of the Duma (Russia's Parliament) after previously working for RT.

There's also the Chinese spy Christine Fang[1] who allegedly had an association with Eric Swalwell, a Democratic US Representative from California and also allegedly had affairs with 2 mayors in the midwest.

It's a pretty safe assumption that other governments are also doing the same thing. They just haven't gotten caught yet.

[0] https://en.wikipedia.org/wiki/Maria_Butina

[1] https://en.wikipedia.org/wiki/Eric_Swalwell#Contact_with_sus...

MilStdJunkie• 2 years ago

Yuuuuuuuup. It's where the infosec term "honeypot" comes from. They had code words in the Soviet union, where ladies were "swallows" and men were "ravens". Back in the day having a "raven" seduce another man was basically espionage jackpot, due to the enormous personal costs of outing the mark. Exceeded only by honeypotting the mark with children, which we might as well use the verb "epsteining". There was also a heyday of absolutely brilliant lady spy-adventurers in the period surrounding the French Revolution - this is right in the wake of Wollstonecraft, remember, what we might call the REAL first wave.

In ancient times sexpionage would flawlessly blend with the world of marriage alliances; Nefertiti, to take one example, was very likely of west Asian descent herself, and was thus a conduit for strange religions and large migrations from Hatti and Canaan. After Akenhaten's death, she probably sent for relatives, which gave impetus to the restoration of Amun, and would definitely put her closer to "spy" than unwitting spouse. I remember, there was one kooky old 19th century book, where Moses was actually Nefertiti. In drag, I guess? I need to go look that up.

Man, I wonder if anyone's written just a general survey of sexpionage through the milennia? Seems like it would sell a couple books.

solardev6• 2 years ago

Pretty girls charged less than Nvidia?

toss1• 2 years ago

It didn't end at the Cold war, at least for the Russians. Most notable is Maria Butina who, among other assignments, was sleeping with Republican political operative Paul Erickson who had close ties to the NRA, and in a relationship with Overstock.com CEO and Trump conspiracy theorist Patrick M. Byrne. She was arrested in July 2018, and charged with acting in the United States as an agent of a foreign government, specifically the Russian Federation. After release, she had a show on Russia Today network and she's now a member of the State Duma representing Kirov Oblast for the United Russia party. [0]

There was also a ring of three Russian women spying on Ukraine, feeding military intelligence to Russia and the Wagner Group. [1] The bust came one day after Ukraine announced it foiled assassination attempt on President Volodymyr Zelenskyy.

And of course there was notorious NY party circuit regular Anna Chapman [2], charged with covertly communicating with Russian intelligence in the largest spy ring bust in the U.S. since the fall of communism.

[0] https://en.wikipedia.org/wiki/Maria_Butina

[1] https://www.businessinsider.com/ukraine-detained-a-group-of-...

[2] https://abcnews.go.com/Blotter/russian-spy-ring-anna-chapman...

saalweachter• 2 years ago

Putin very much has taken the position that the Cold War isn't over til he says it's over. You can't lose the game if you refuse to stop playing.

almatabata7• 2 years ago

Reminds me of that east german operation where they sent out "romeos" to seduce women in west germany (https://www.cia.gov/stories/story/romeo-spies/).

carabiner• 2 years ago

As depicted in the tv show The Americans. Best show of all time.

livrem• 2 years ago

https://foundation.mozilla.org/en/privacynotincluded/article...

api5• 2 years ago

It's even more dystopian than Bladerunner 2049. From the film it seemed like Joi was a local model that he could even download to his own device.

It's really depressing to me as someone who grew up with the tech optimism of the 80s and 90s to watch how this whole area has transformed into a horror show. Everything since the PC revolution has been a net negative for regular people: mobile devices (addiction, manipulation, mental health crisis), cryptocurrency (gambling, pump and dump scams, ransomware), cloud (no privacy, no data ownership, perpetual rent), streaming (DRM, paltry royalties to artists), and now AI (manipulation, un-filterable spam, political propaganda).

All these technologies are used in very positive ways too, but it seems like that's only for the highly tech-savvy or those with money. If you are neither tech-literate nor wealthy they're weaponized against you.

The ethos of the industry seems to be: if you aren't smart or rich, not only are you fair game but you deserve it. That's the vibe I get from the hyper-elitist ideologies that have proliferated in this culture. I'm talking about things like neoreaction and related ideas.

As a tech person myself I feel like I'm now part of some kind of corrupt wicked black magician priesthood destined to rule over the rest of humanity. If I extrapolate forward to where this is going I see a future where most of humanity is fully addicted to things like infinite feeds, VR games, and AI girlfriends and controlled by misinformation and psychological manipulation. Essentially the bulk of humanity is given an API whereby it can be programmed and marshaled as a service by a mix of the rich, governments, and criminals.

This is exactly cyberpunk. Cyberpunk was by far the most prophetic genre of sci-fi. It achieved this by being optimistic about technology but pessimistic about human beings.

01HNNWZ0MV43FF5• 2 years ago

Misinformation, maybe. But if people really get addicted, it'll hurt the economy, and then the invisible hand will step in. That's why opiates are illegal

api5• 2 years ago

I don't see that. We've given up on the idea that human beings have any value beyond their impact on GDP, so if the lower classes aren't seen as necessary they will be allowed to die on the street.

Addicting them to cybernetic control systems might be seen as a better alternative because it'll allow them to be "monetized" in some way-- make them go get gig jobs, then gamble away their earnings online.

You can see this future already in some cities like San Francisco and across the poorer parts of rural America.

https://kagi.com/proxy/2023-Drug-od-death-rates-1.jpeg?c=pqA...

The future is here. It's just not evenly distributed.

llm_trw• 2 years ago

The opioid epidemic is very much legal and very much _not_ being stopped.

suoduandao34• 2 years ago

AI girlfriends solve the 'angry single men' (AKA drone AKA incel) issue even better than pornography - and I've seen a lot of public health types touting the benefits of pornography.

Miraste• 2 years ago

Laws banning products is the exact opposite of the invisible hand.

beretguy• 2 years ago

It's even more dystopian than Bladerunner 2049. From the film it seemed like Joi was a local model

I believe she was connected to Internet and was spying on K for Love, even more so when in his pocket.

malermeister• 2 years ago

I fully agree with this take. Yanis Varoufakis also talks a bit about this technofeudalist world our "industry leaders" are trying to create:

https://www.theguardian.com/world/2023/sep/24/yanis-varoufak...

“Imagine the following scene straight out of the science fiction storybook,” he writes. “You are beamed into a town full of people going about their business, trading in gadgets, clothes, shoes, books, songs, games and movies. At first everything looks normal. Until you begin to notice something odd. It turns out all the shops, indeed every building, belongs to a chap called Jeff. What’s more, everyone walks down different streets, and sees different stores because everything is intermediated by his algorithm… an algorithm that dances to Jeff’s tune.”
It might look like a market, but Varoufakis says it’s anything but. Jeff (Bezos, the owner of Amazon) doesn’t produce capital, he argues. He charges rent. Which isn’t capitalism, it’s feudalism. And us? We’re the serfs. “Cloud serfs”, so lacking in class consciousness that we don’t even realise that the tweeting and posting that we’re doing is actually building value in these companies.

analognoise• 2 years ago

As a tech person myself I feel like I'm now part of some kind of corrupt wicked black magician priesthood destined to rule over the rest of humanity

It’s funny to hear the worker bees think this, while all the data and code is really owned by the emergent mega corp. I guess it’s easier to swallow psychologically that you’re part of the secret cast of rulers rather than a cog who is subject to the rubber hose treatment at the slight change in a law like the rest of the workers.

For a group so supposedly smart, a little class consciousness would go a long way.

jerf• 2 years ago

As my age slowly starts to creep into the earliest phases of "Old Man Yells At Cloud", I find that I'm not distrusting new technologies because they're just ambiently bad because I didn't grow up with them. I find I'm distrusting new technologies... and a generous helping of old ones too... because I don't trust the people behind them.

Not quite the same thing as the traditional "I just don't like anything new".

A good AR toolset sounds like fun. But am I so arrogant as to believe I can resist all the "nudges" that will inevitably be applied to everyone using the tech? There's simply no way the nudges are for my benefit; claims that they are are just part of the marketing campaign.

It's seriously annoying because I'd love to play more with these technologies and figure out how to use them. But it's clear rather a lot of them come with implicit clauses in the contract I'm not willing to sign. I'm not interested in running my political opinions past Silicon Valley AIs. I've got a finite amount of tolerance to poke through with-malice-aforethought uses of addiction mechanics against me to try to figure out how to use technologies to my advantage. And so on. The more amazing the technology, the shittier the deal on offer it seems.

api5• 2 years ago

This isn't "old man yells at cloud." This is a subset of a larger realization that dawns on all engineers over time:

A lot of the things you thought were technology problems are actually people problems.

I've come to this realization about a lot of what's bad about the Internet's architecture. It's not because we can't do it differently. It's because there are human social and especially economic incentives that make it this way, and introducing better tech won't change those.

poisonborz• 2 years ago

Who planned and developed these products? The same generation of developers you talk about.

autoexec5• 2 years ago

All these technologies are used in very positive ways too, but it seems like that's only for the highly tech-savvy or those with money. If you are neither tech-literate nor wealthy they're weaponized against you.

Honestly, even if you are wealthy and highly tech-literate they are still weaponized against you. You literally can't protect yourself and still use some products, and some (like the cell phone you carry) are too useful to be avoided. I go out of my way to avoid as much of it as I reasonably can, but technology is leveraged against us everywhere. You can never avoid all of it and still participate meaningfully in society

skocznymroczny• 2 years ago

Emphasis on "your". My AI girlfriends are running locally on my GPU and don't require internet access to function.

steelframe• 2 years ago

My AI girlfriends are running locally on my GPU

The future is amazing.

toomuchtodo2• 2 years ago

Apple positioned well for this with iOS 18. AI girlfriend can run on your Homepod!

belval• 2 years ago

Despite the obviously dystopian undertones, it's truly an interesting time to be alive.

arkh• 2 years ago

I remember a time when dating websites were for nerd losers. Now they're the norm.

I can't wait to see how things are 20 years from now.

jordanb• 2 years ago

Love is a warm GPU

jjmarr• 2 years ago

The example waifu in text-generation-webui is good enough for me.

https://github.com/oobabooga/text-generation-webui/blob/main...

solardev6• 2 years ago

don't require internet access to function

I can't even say that about my real girlfriend. Take away the internet for a few hours and see what happens...

lagniappe2• 2 years ago

I hope you're joking.

cjbgkagh2• 2 years ago

So much of my girlfriend’s behavior is driven by TikTok trends that most often it feels like I’m dating TikTok and not an independent person.

lebean• 2 years ago

My waifu is the word suggestion Markov chain on my phone. Conversation has gotten dry, but we make it work.

coldtea4• 2 years ago

Maybe you let it go dry man. You need to put in more effort.

I have the same phone word suggestion Markov chain waifu and we couldn't be closer: we complete each other's sentences!

robotnikman• 2 years ago

Not your GPU == Not your AI Waifu

DonHopkins3• 2 years ago

Do you AI girlfriends know about each other? You could be in a lot of trouble if they find out you've been cheating on them if you're not honestly and openly polyamorous with them.

Players of The Sims video game try to maintain multiple wives in different bedrooms, romancing and woohooing them behind closed doors, so they didn't get angry at each other.

After developing The Sims at Maxis and releasing it into the world in 2000, I made some tools to enable players to create custom content, and worked with some indie Sims fans to make the Simprov Wedding Playset for The Sims 1. We wanted to "woke it up" to support same sex marriage and non traditional customs and ceremonies, before Maxis/EA officially supported it in later versions of The Sims.

It includes a "Cupid" cheat object that helps you instantly and effortlessly fall deeply heads-over-heels in love with any other Sim in your neighborhood you wanted (including dogs and cats)! That made it much easier to get on with the next steps in planning your dream wedding, by fast-forwarding over all that pesky romancing and dating and getting to know one another nonsense. Cupid puts the heat into cheat.

Speed Dating with Cupid:

https://www.youtube.com/watch?v=YVUP9OXmHTM

The Cupid was based on reprogramming and reskinning the original phone, so clicking on it popped up a pie menu of every family in the neighborhood you could call, with submenus for every member of each household (including the pets) you could fall in love with.

To stress test it, I systematically fell in love with one person after another.

Selecting a Sim first invites them over. They immediately arrive on the lot, and you have to greet them. Then they walk over to the Cupid, which makes them instantly fall in love with you.

Then they walk over to you, give you a passionate kiss, while a sting of romantic music plays, and hearts and "plus relationship" icons flash above each of your heads, showing that you've each just fallen in love.

What happens next is up to you the player, or the autonomous characters themselves, if you don't interfere with their lives by clicking from pie menus to control them. It's like a pinball machine with pie menu flippers and multiple hearts in play.

So I continued to stress test the Cupid by inviting one Sim after another over to fall in love with me. If I didn't wait for my previous lover to leave, or lock them in a room and remove the door, or tempt them into the swimming pool and remove the ladder, and then fell in love with another Sim in the same room right in front of them and they saw us kiss, cheating in front of them piss off the previous lovers, and they queue a "Slap in Face" interaction, which decays our relationship.

The ultimate effect of falling in love with one Sim after another is that all your previous lovers would hang around getting madder and madder at you, and each time you kiss a new lover, all your previous lovers in sight would line up to slap you one by one (they're programmed to be too polite to all slap you at once, so they queued and take turns). So your relationship with your first lover decays from the most number of slaps, and so on, up to your perfectly head-over-heels relationship with your most recent lover.

The challenge is to see how many consecutive slaps you can get from scorned lovers. (Hint: put out a buffet table and lots of port-a-potties so they can eat and shit and don't have to go home. Or use the Buddha cheat object to keep everyone happy with their stomachs full and bladders empty, which makes long drawn out wedding ceremonies go smoother.)

One solution to this dilemma is to install the ZombieSims expansion playset, and zombify your previous lovers before moving on to the next.

https://news.ycombinator.com/item?id=34485103

ZombieSims is a mind-blowing, brain-eating, tour de force Sims 1 fan expansion pack by two of the greatest Sims user created content artists and programmers: Heather "SimFreaks" (who created SimFreaks.com and much of the beautiful content for Sims 1 at http://www.simfreaks.com/index.php including themed play sets like http://www.simfreaks.com/themes/storytime/pirate/index.shtml ) and Steve "SimSlice" (who created SliceCity: SimCity within The Sims at http://simslice.com/Slicecity.htm by programming many interlocking objects in SimAntics, and also many other amazing Sims 1 objects like the weather machine at other cool stuff at http://www.simslice.com/Objects-Electronic.html ).
Heather and Steve were both early Sims 1 fans who each published their own popular web sites with downloadable objects, met through the Sims 1 modding community, then eventually moved in together and got married, and now they've combined their extreme art and programming talents to make an intricately intertwingled collection of Sims 1 Zombie objects, with a whole lot of original artwork and programming!

https://zombiesims.com/

Twitch streaming videos:

https://www.twitch.tv/simfreaks_heather/videos

Highlight: Zombie Sims - Beta - Everybody Dies

https://www.twitch.tv/videos/1049524221

Check out some of the crazy menus that pop up -- this demo barely scratches the surface!

Here is a video of Simprov (unfinished but playable with programmer art and sample text), and the zip file with a huge collection of The Sims 1 objects, including the Cupid and many other Simprov Wedding Playset objects, so you can try reproducing the results of my experiment in the privacy of your own home.

Simprov Wedding Play Set

https://www.youtube.com/watch?v=Mwt5LJlrMe8

https://news.ycombinator.com/item?id=34492255

(In Professor Farnsworth's voice:) Good news everyone!
I asked Heather permission, and she says it's OK for me to give away the huge collection of custom Sims objects I have that includes an archive snapshot of many classic SimFreaks objects, as well as all the unreleased SimProv Wedding Playset objects that Heather and Donna and Steve and I created years ago but never finished and released, and a whole bunch of other stuff like the Transmogrifier object that randomly changes your body, the Dumbold voting machine that sometimes makes you accidentally vote for Pat Buchanan, Satan who shows up when you're depressed and offers to buy your soul, the Crowd Sitter that makes everyone gather together and sit down on chairs, and the Cupid that lets you instantly fall in love with anyone in the neighborhood, and the Buddha that makes everyone happy and not piss themselves and fall asleep in their own puddles of urine during parties.
I don't have time to actually support and debug any of this stuff, but at least I recently updated the Cupid to be compatible with the Pets expansion pack, so it now lets you fall in love with any pet in the neighborhood. (You just can't actually marry them -- not that there's anything wrong with marrying cats and dogs, but we didn't have the animations for that!)
If you want to express your appreciation, then please subscribe to Zombie Sims for a $9.99 lifetime membership, and then you can play around with inviting lots of Zombies to your weddings and see how that works! (Or don't invite them, and they will crash your wedding anyway!) But no guarantees or warranties that it doesn't devolve into a bloody mess!
Here's my special collection of Sims 1 downloads, including the unreleased and not quite finished "SimProv" wedding Playset and handy "Cupid" that lets you instantly fall in love with anyone in the neighborhood (including pets)!

https://donhopkins.com/home/DonsSims1Downloads.zip

    cd ~/Downloads
    curl https://donhopkins.com/home/DonsSims1Downloads.zip > DonsSims1Downloads.zip
    cd ~/Applications/Wineskin/The\ Sims\ Complete\ Collection.app/Contents/SharedSupport/prefix/drive_c/Program\ Files\ \(x86\)/Maxis/The\ Sims/
    unzip ~/Downloads/DonsSims1Downloads.zip

[...]

Transmogrify Self: A quick demo of The Sims Transmogrifier personified in The Sims 1. Graphics by SimBabes, programming by SimSlice.

https://www.youtube.com/watch?v=dsTbs7IL5EI

To find the Cupid and other Simprov items, go into buy mode, press the last icon of three dots for "Miscellaneous", then press the first icon with a pool table for "Recreation". The main item of the Simprov wedding playset is the "Hope Chest", which has a "Help" item that explains what to do next, and it summons a wedding consultant (who you can dismiss and call back if you don't like her hanging around in your bedroom forever). Then you can click on the hope chest to make other objects like the Cupid, and click on the wedding consultant to make catalogs of other items (most of them are just placeholder programmer art right now, but some of then configure things like what kind of wedding you will have and who will officiate it), but the idea was that you could order lots of items through the catalogs that you couldn't get through the normal shopping interface. But for now most of the wedding items are still in the build mode shopping catalog. The Simprov Wedding Playset video above walks through how to use most of the objects!
Also be sure to check out Donna's beautiful wedding beds, the luxurious buffet with ice dolphin sculpture, gold inlaid glass dining table, fancy dollhouses, elegant dolls, and many other premium objects identified as Simprov, SimBabes, and SimFreaks in their catalog descriptions.

OfficialTurkey• 2 years ago

What

dr_kiszonka• 2 years ago

It is clearly a passion project, innit?

Arrath• 2 years ago

I just have to say that I love your posts.

MikeTheRocker• 2 years ago

Not your compute, not your girlfriend, as they say.

moose44• 2 years ago

Same applies to ChatGPT.

tivert• 2 years ago

“To be perfectly blunt, AI girlfriends and boyfriends are not your friends,” said Misha Rykov, a Mozilla Researcher, in a press statement. “Although they are marketed as something that will enhance your mental health and well-being, they specialize in delivering dependency, loneliness, and toxicity, all while prying as much data as possible from you.”

I can't wait for AGI and all the good it will surely do for humanity in the hands of capitalists! Their marketing copy gives me hope for the future!

zooq_ai2• 2 years ago

Yet another manufactured outrage. Nobody cares about your 'individual' data unless you are in the 10,000 important / hot people in the world

pimlottc• 2 years ago

That's becoming less and less true. AI makes it easier than ever to automate personalized scamming.

zooq_ai2• 2 years ago

It's a different problem that needs to be solved

AlexandrB2• 2 years ago

Nobody cares until you say something dumb online. Then everybody cares very intensely for a little while. Having all this data out there is a growing liability for the average person.

kanobi3• 2 years ago

It's always ironic to find these kinds of articles on sites that have hundreds of background trackers. Even though I "rejected all" unnecessary cookies I still see around 16 of them coming through. Like adsafeprotected, adlightning, google-analytics.

rchaud• 2 years ago

Go to the direct link then: https://foundation.mozilla.org/en/privacynotincluded/eva-ai-...

kanobi3• 2 years ago

I mean the point of the article still stands, it was just an observation of how funny it seems.

I have ublock installed, so this can be at least prevented with news websites. Which can't be said for unseen tracking in chat-bots. Collecting and storing this data without consent obviously sidesteps GDPR, so I'm not even sure about legality of these practices in EU.

OJFord2• 2 years ago

I would not have guessed this was a big enough thing that it's an entire category: https://foundation.mozilla.org/en/privacynotincluded/categor...

pjc50• 2 years ago

I would. It's probably a pretty big but otherwise invisible hikikomori market.

OJFord2• 2 years ago

So a target market of so far (some subset of) an estimated 1M Japanese people? (From Wikipedia summary of the term.) I still would not expect that to be something that multiple companies cater for, enough that Mozilla has privacy-audited 11 of them and created a whole companies-we've-privacy-audited category for it.

burningChrome• 2 years ago

Wow, just like. . . .WOW.

Mozilla found the AI girlfriend apps used an average of 2,663 trackers per minute, though that number was driven up by Romantic AI, which called a whopping 24,354 trackers in just one minute of using the app.

coldtea4• 2 years ago

What's a tracker in this context? Distinct parties interested in the data?

charcircuit• 2 years ago

and what exactly was sent? Sending personal information is much different than sending what pages you opened on the site (view counter).

sgift• 2 years ago

I think this is the article from *PNI Gizmodo is referencing: https://foundation.mozilla.org/en/privacynotincluded/article... - and here's the detail page for the AI Bot in question: https://foundation.mozilla.org/en/privacynotincluded/romanti...

All I could find was this:

As part of our research, we looked to see how many trackers the app sends out when you use it. Trackers are little bits of code that gather information about your device, or your use of the app, or even your personal information and share that out with third-parties, often for advertising purposes. We discovered that Romantic AI sent out 24,354 ad trackers within one minute of use. That is a LOT of trackers (for reference, most apps we reviewed sent out a couple hundred trackers). Now, not all these trackers are necessarily bad. Some might be for legitimate reasons like subscription services. However, we did notice that at least one tracker seemed to be sending data to Russia, whose privacy laws aren't necessarily as strong as those elsewhere.

So, probably requests to third parties?

jncfhnb4• 2 years ago

Probably something like a keypress event?

gkbrk• 2 years ago

Who presses 24 thousand keys a minute?

ptek2• 2 years ago

That's a lot of bandwidth :(, especially for people who don't have unlimited data on their cell phones or have wifi at home.

racl101• 2 years ago

[pikachu face]

psacawa• 2 years ago

Technoparasitism. In my opinion, the very premise of AI girlfriends veers so far away from human flourishing that privacy concerns seem entirely secondary.

autoexec5• 2 years ago

I don't understand the hand-wringing over it (excepting the exploiting lonely people and collecting massive amounts of highly sensitive data part which is obviously evil). Nobody is going to reject a human partner because they already have a cell phone app. For people who would otherwise never have a human partner why not let them have an app if it makes them happy?

JohnFen3• 2 years ago

Nobody is going to reject a human partner because they already have a cell phone app.

I'm not so sure about that, actually. Most people won't, but there do exist people who don't do what's needed to have a human partner because they have porn instead. I don't see how an AI girlfriend would be any different.

why not let them have an app if it makes them happy?

Who's talking about not letting them have the app? I find it incredibly sad, but it's not my life so not my business.

reducesuffering• 2 years ago

Treating it as "just having a cell phone app" is quite the trivialization of how much time and emotional investment could go into it. Sort of like calling someone who plays video games 10 hours a day "just on the computer like everyone else at work." There are obviously many people in the modern world that are so engrossed in screens like Youtube, Instagram, and video games, that it's coming at the expense of their in person interaction and relationships.

Personally, I agree with GP, that it's quite a terrible travesty that people are being sucked into being controlled by a machine and forgoing the eternal human condition of building a relationship with real human people in person like we always have.

jstarfish2• 2 years ago

For people who would otherwise never have a human partner why not let them have an app if it makes them happy?

I want to agree with you but my optimism has rarely reconciled with history itself.

There's a fallacy here: why are we writing off an entire subset of the population as irredeemable? Ugly people find love. Even assholes find love. Weight is not an exception, nor disability.

This is like giving heroin to someone that's depressed. Sure it'll make them happy, but it won't end well.

For something that takes place behind closed doors there's no oversight either, which is a problem enough with rogue therapists (see the Ramona False Memory case). This undermines social cohesion and makes radicalization trivial-- nobody would see it coming at all enough to intervene.

We're talking private reinforcement of ideas like kill-your-parents-and-yourself, or commit-treason-and-sabotage type of suggestions. Nobody knows little Jimmy is jerking it to Israeli propaganda.

https://www.lipstickalley.com/threads/pornography-transgende...

I've asked before why there is no hypnosis porn to make people fetishize anything other than feminizing men. There's no "you like natural beauty" hypno, or "you're fine just the way you are" hypno. It's all an ad campaign to specifically drive sales of therapy and hormones. One nation produces 90% of the world's supply; guess who.

All of this to support a commercial endeavor. God help us when this sort of personalized influence is weaponized for military use.

nonrandomstring• 2 years ago

Right now isn't the entirety of AI - that scours the internet for other people's content for training - parasitical upon humanity?

udev4096• 2 years ago

Nothing new. Every single service, especially associated with "AI", are meant to collect data about you. Honestly, they fucking deserve it. How can you be so fucking stupid?

teekert2• 2 years ago

Im always reminded of the ending of the movie “Her”. I won’t spoil it here but it’s a masterpiece, it was ahead of its time.

mtlmtlmtlmtl• 2 years ago

Kind of reminds me of West World too.

gettodachoppa2• 2 years ago

IIRC, the AI "girl" becomes able to communicate with other people, and she instantly cheats on him like 2000 times in 1 day and then breaks up with him and leaves?

I think the movie tried to frame it as the right ending but to me it felt like the worst of MRA talking points about women's nature, validated on the screen.

teekert2• 2 years ago

*spoiler alert*

I think “she” was serving an enormous amount of people from the very start and the main character simply made the mistake of projecting his human values onto something completely alien.

As we do now with our chatty buddies.

I find it kind of concerning that you somehow link the AI presented in “Her” to our human women. You fell for it too I’d say.

In the end, the AI never lied. “She” simply followed “her” programming and took the main character for an emotional ride. Would you have commented the same if the genders of the customer and the AI agent were reversed? If you wouldn’t have been listening to that sweet voice of Scarlett Johansson?

The movie is 10 years old. It’s a masterpiece. It makes you think. It makes you look at your own reflection in the black screen as the movie fades out.

FrustratedMonky2• 2 years ago

Imagine if sex dolls are 'cloud enabled' what data they would gather.

nickthegreek2• 2 years ago

https://nypost.com/2024/02/12/business/woman-sues-adam-and-e...

fransje26• 2 years ago

"Your thrust to climax ratio is low."

booleandilemma• 2 years ago

And your waifu is trash.

sagebird• 2 years ago

Is there anything intrinsically privacy hostile about this potential industry, beyond sending data to a server - which is something you might do if you engage with doctors or therapists virtually?

I hope that consumers choose to pay a premium for privacy preserving services. If users are indeed getting alot of value from the arrangement - I would hope they don't use free or cheap services that need data harvesting as part of the revenue.

My concern is that even premium products would want to pool user interaction logs in order to train better models - which isn't as directly hostile as packaging user labeled data and selling it, but it is a sloppy art to claim you are anonomyizing user data. As any sufficient anonymization necessarily destroys information that would be useful to training.

mablopoule2• 2 years ago

Imagine your therapist and your significant other had a mailing list where they expose all the confidences you said to them.

If that's not privacy hostile, then honestly what is?

kanobi3• 2 years ago

But the problem is how can consumers really know if the company is selling this data and to who, without some kind of legislation? Being a premium product is just an indicator, but it's not definitive.

I think even CCPA in California should be able to prevent abuses like that. At least you should know what data is being sold and you should be able to opt-out. If that's really the case, only time will tell.

autoexec5• 2 years ago

Is there anything intrinsically privacy hostile about this potential industry, beyond sending data to a server - which is something you might do if you engage with doctors or therapists virtually?

Presumably people share things with their romantic partners that they don't share with doctors and therapists (assuming they aren't dating one). People shouldn't be using virtual doctors or therapists for the same reasons though. Everything they reveal about themselves will be collected, analyzed, stored forever, leaked/sold, and ultimately used against them at every opportunity.

It's also basically a myth that only free or cheap services abuse your data. Paid services and extremely expensive products do it all the time too. There is no company that wouldn't make a greater profit by taking your money and then also abusing your data so they pretty much all do it. The only services you can really trust are the ones you can run locally that don't send your data anywhere.

squigz• 2 years ago

I would hope they don't use free or cheap services that need data harvesting as part of the revenue.

Because paying for a service means your data won't be sold?

renewiltord• 2 years ago

The set of people with AI Girlfriends and the set of people who care about their data being harvested have a null intersection. Who honestly gives a fuck. Privacy advocates are constantly freaking out about stuff. Meanwhile my medical records and DNA sequences are publicly available data because I donated them to a public database. And guess what? I'm fine.

maxglute• 2 years ago

Im surprised google hasn't made this a feature / selling point. I'd pay 20 bux for 2tb and an ai waifu. And be irrationally mad when google kills her. Top post has a point, AI waifu + cloud = bad news.

rsynnott5• 2 years ago

though that number was driven up by Romantic AI, which called a whopping 24,354 trackers in just one minute of using the app.

Coming soon: the world's longest GDPR cookie consent sheet.

ptek2• 2 years ago

Do AI girlfriends ask computer security questions? i.e, I like cars, what was the first car you owned? Tell me about yourself, Tell me about your mother, what was your mother's maiden name? I wish I could touch you, can you put your finger on the fingerprint scanner so I can touch your hands?

Jiocus• 2 years ago

"Your girlfriend is just someone else's computer"

Your AI Girlfriend Is a Data-Harvesting Horror Show

Related Stories