A failed AI girlfriend product, and my lessons

https://mazzzystar.github.io/2023/11/16/ai-girlfriend-product/

by mazzystar • 3 years ago

253 382 3 years ago

mazzzystar.github.io

You should not even try to create these sorts of technologies. Take a break from technology, spend time outdoors with plants and animals. These types of technologies will only lead to a worsening of this already sick society.

HPsquared4• 3 years ago

Could say the same about antidepressants. "Fix the causes, not the symptoms!" And yet, here we are.

vouaobrasil6• 3 years ago

I agree, 100%. Here we are indeed. But that doesn't mean we should not try and change things. One step at a time, right?

mejutoco• 3 years ago

Saying that something exists is not an argument about why something _should_ not exist.

But I agree, "should" discussions are rarely the most productive. Shoulda, woulda, coulda, I guess :)

throw18376• 3 years ago

antidepressants can help people fix root causes. take them for 6 months to 2 years or so, break out of a bad cycle, get some space without depression symptoms to build up healthy habits and restore a social life.

there are some bad profit motives to keep people on SSRIs indefinitely. however at least they don't compete with other aspects of your life. and now that they are dirt cheap (at cheapest, $4 for a month supply at Walmart) the profit motives are much less than before.

hard to imagine an "AI girlfriend" that could be similarly short-term and bring positive changes. it's just dystopian. the most profitable way to run such a product is not to help, it's to eat up as much user time as possible and outcompete healthy activities and real human relationships.

phatskat5• 3 years ago

“Fix the causes”

How, exactly? Yes, some people experience depression as a symptom of something than can be addressed through various therapies, but what about people who just have it wired into their brains? How do we “fix” that?

InfiniteRand• 3 years ago

I think it's helpful to look at depression as a group of diseases like cancer. There are some cases of depression where we know the root cause: thyroid issues, sleep apnea-related depression, low testosterone related depression, brain tumors, etc. In those cases, addressing the root cause is probably the best treatment method. This is similar in some ways to cancer, where certain cancers have very definite causes and can be treated by dealing directly with those causes (like cervical cancer).

Then there are all the other cases, some of which likely have simple causes that are just unknown, some of which have complex causes that are intermeshed deeply with the biology of human thinking. In that case, you have blunt tools, like anti-depressants, to address the pathway of the disease without knowing the exact cause, much like chemo-therapy can be used to attack cancer cells without knowing what prompted the cancer originally.

So in short, yes treat the causes, but when you don't know the cause it's not irrational to treat the symptoms as best you can

krzat4• 3 years ago

If the AI had a secondary goal of improving customer's mental health, it could work very well. Usually you can't fuck your therapist.

vouaobrasil6• 3 years ago

I think in theory, you are right. However, because AI is so easily accessible and it is being developed in our current, highly commoditized society, there is really no change that it will ever only be used for good in this way.

chpatrick• 3 years ago

You can't really fuck your imaginary shoggoth girlfriend either.

SoftTalker• 3 years ago

Usually you can't fuck your therapist.

Isn't that what prostitution is though?

bambax2• 3 years ago

Who are we to judge the users of such a product?

Some people suffer from a variety of disabilities which makes them unable to find friends and engage in meaningful relationships with others, etc.

vouaobrasil6• 3 years ago

I am not judging the users of the product. I am saying that the overall product will worsens society.

optimalsolver2• 3 years ago

spend time outdoors with plants and animals

Which animals? Blood ticks?

pixl972• 3 years ago

Bot flies. Or a bunch of those things we've driven extinct because they tried to eat us.

Naturalistic fallacies are common here :/

icepat3• 3 years ago

Had a similar moment with playing around using smart contracts. As a joke I wrote a crypto for the company I worked at, which then got added to employment contracts, and an internal DAO styled administration layout.

After coding this out for a few hours I realized I had re-created the company scrip, except on a blockchain. I ended up turfing the project and spending some time thinking about how horrific a future technology like this could create.

Dylan168074• 3 years ago

I don't get it. Yes, you re-created scrip. That's the opposite of a new technology coming in and causing problems, isn't it? It's a known issue with a known solution.

kobalsky• 3 years ago

I don't know why studies are even being made on the effects of these novel technolgies. We should just point the scientists to your email for answers.

vouaobrasil6• 3 years ago

Studies by scientists are typically within the framework of modern western society. And moreover, if you actually look at some studies done by psychologists, you will find a large body of literature showing that it is mostly detrimental.

kridsdale32• 3 years ago

Not a lot of Hackernews posts are so depraved that they get a "Jesus Christ, touch grass" response.

Der_Einzige3• 3 years ago

God forbid that ugly people get to have the same kinds of positive relationship experiences that attractive people get. I can’t believe how the moment that a technology comes out which gives incels real options it’s seen as being terrible for society.

It’s awesome for society.

mrguyorama4• 3 years ago

This doesn't give an incel an "option" anymore than playing mario kart makes you a better race car driver. As a business, they will be optimized to fit an incel's wants, not to help them adapt to reality and healthy relationships.

Incels need therapy and support, not a fake "girlfriend" program that charges them money to reinforce their existing horrific worldviews

krapp• 3 years ago

Plenty of ugly people do have those relationship experiences, all the time.

But I guess if something like this keeps incels from spree shootings then it's a net win for society.

mrshadowgoose5• 3 years ago

Agreed! It would be a real shame if lonely unattractive people had a chance at something resembling a romantic relationship. /s

Contrary to what one would assume, I don't have this problem. However I do work in the IT field, and am surrounded by perpetually sad and lonely people who have never experienced romance. It's really really sad. And no, none of these people are the "angry hateful incel" type that the internet loves to dismiss and villainize.

If you aren't attractive, your chance of finding romance is low. This is just a cold and ugly fact of the human condition. There simply isn't "someone for everyone".

Why should we disallow lonely people from having something, even if it's simulated?

technics256• 3 years ago

I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.

This was an excellent read. And I applaud the creator for experimenting with this to the fullest.

swyx• 3 years ago

mustve been hard to admit publicly. i wish the author hadnt edited it out subsequently. just own it bro. you wanted to understand your customer.

Traubenfuchs2• 3 years ago

So, all in all, OpenAIs puritanism ruined this product, made the developer sad and took away a virtual porn toy from lonely, horny men. Who were they doing this for? Who is the winner? They lost a customer and made a lot of people sad.

immibis• 3 years ago

You aren't the customer, you're the product.

Traubenfuchs2• 3 years ago

People paying for their API must be their customers, I mean, if there is no one paying for their API, how do they make money? (Besides ChatGPT+)

FeepingCreature• 3 years ago

Who were they doing this for?

VISA probably. Card processing is weirdly puritanical.

papa-whisky• 3 years ago

I've read that payment processors don't like dealing with porn companies because of the very high rate of chargebacks they attract, not because the processors are Puritans...

anthony100• 3 years ago

Interesting. Not sure how this is different from Onlyfans and lots of money is going there.

mensetmanusman4• 3 years ago

They want chatgpt in schools to own the next gen of workers.

feintruled2• 3 years ago

Well this went places. The dev really 'ate his own dogfood' so to speak. I thought it was going to be a despairing look at how his companion AI was continuously turned to sleazy uses by a pervy userbase but that's pretty much what he was going for it seems. I have to say, I hated this part: "...gained a lot of Chinese users. I immediately realized the political risk: users could set Dolores’ background, personality, and attributes, which might lead to political issues. I quickly took Dolores off the shelves in China."

Just pre-emptorily censors himself to appease the Chinese government. They've done their job well.

Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint.

globular-toast4• 3 years ago

Also it seems like he had full access to the replies via the voice generator, which sure isn't great from a privacy standpoint.

Whose privacy is being violated? Dolores's?

feintruled2• 3 years ago

You joke, but I can forsee issues. If he has the replies to a particular user, and he has a means of IDing that user, he could cause them a lot of hurt. They are getting intensely sexual replies back, they aren't getting that without supplying sexual prompting. Could also be used to out gay users.

cnity• 3 years ago

Have you never listened to one half of a phone call before? You can glean a great deal of information about the person on the other end.

butlike3• 3 years ago

At the risk of sounding like a space case: I think we need to make sure we know what consciousness is before we make a bunch of abusable models.

I, for one, wouldn't necessarily like to be a depraved-sex slave bot if I was able to reason about it.

scyzoryk_xyz2• 3 years ago

Pretty much came into the comments to agree with the self-censorship part there.

But then the author did use 'Her' as the ideal example. That's a pretty good indicator they didn't get the point.

persnickety2• 3 years ago

What's the point?

luke-stanley• 3 years ago

Their name is "Ke Fang", and they say Chinese is their first language...

firexcy• 3 years ago

And the post was translated into English with GPT according to their "about" page. That's why mouthful and unnatural phrases scattered between the lines.

whywhywhywhy• 3 years ago

Just pre-emptorily censors himself to appease the Chinese government.

Every major tech company does the exact same for China, Russia and UAE. Business as usual

narinxas• 3 years ago

every major business company has also done this for the USA... not just tech companies

so I agree, business as usual

telotortium• 3 years ago

He's Chinese - makes sense to pre-censor himself, unfortunately.

phreack2• 3 years ago

I hadn't read anything like that so I just checked and searched for "China" and it's not in the article. Looks like he censored the censorship.

Edit: Yeah, looking at an archived version, it's been edited quite a bit.

DryLabRebel• 3 years ago

Lol, I did the same thing.

Everything about this article and this comment section has started my Saturday off on a 'humans bug me' tangent.

FartyMcFarter• 3 years ago

This guy created and lived a whole Black Mirror episode by himself:

I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.
However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.

sva_• 3 years ago

Maybe he shouldn't have named it after the most hated Harry Potter character.

skummetmaelk• 3 years ago

He was inspired by Westworld.

amelius2• 3 years ago

More likely:

https://en.wikipedia.org/wiki/Dolores_Abernathy

HPsquared4• 3 years ago

I always think of that Seinfeld episode.

amelius2• 3 years ago

Well, the meaning of the name is the plural of "sorrow" or "pain".

So he had it coming :)

https://en.wikipedia.org/wiki/Dolores_(given_name)

nwoli• 3 years ago

Never get high off your own supply

red-iron-pine2• 3 years ago

This was drifting away from the essence of ‘Her’.

"choke me with that dead cat"

(note: dead cat usually means some sort of microphone cover)

ReactiveJelly7• 3 years ago

I think he edited that line out when it got big on HN lol. And clarified that she was named for Westworld, not... "She was Dolores on the dotted line"

Michelangelo11• 3 years ago

Yeah, I read the comments and I was like damn ... that's not in the linked version.

averageRoyalty• 3 years ago

Yeah, I think he's edited a few things. A lot of comments here no longer make sense.

Here's the original:

https://web.archive.org/web/20231116091926/https://mazzzysta...

Octokiddie2• 3 years ago

Looks like that part was edited out.

bloodyplonker22• 3 years ago

It was too cringey and creepy.

bitwize• 3 years ago

However, I started to feel a sense of loss: if every Dolores user was engaging in anonymous, NSFW role-play, what real significance did it hold for me? This was drifting away from the essence of ‘Her’.

That is indeed exactly the essence of Her. Had the author watched the movie to the end?

globular-toast4• 3 years ago

Yeah, it's like the author read the blurb for Her and didn't even read a synopsis.

theGnuMe• 3 years ago

Or Blade runner 2049

laurent_du• 3 years ago

Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale. Depressing stuff but this is the world we live in. Thank you to the author for this interesting read.

louthy• 3 years ago

Eventually, a virtual girlfriend app that will put together sound and image will be released an empty the pockets of sad, lonely men on a large scale.

Sounds indistinguishable from an actual girlfriend ;)

orng• 3 years ago

Although probably intended as a joke, this is perhaps the saddest comment I have read on HN.

phatskat5• 3 years ago

[flagged]

wruza2• 3 years ago

You mean snatch their money out of hands of twitch streamers. It’s not in their pockets already.

em500• 3 years ago

If the barriers to entry to the tech are low, virtual girlfriends should not be expensive. Probably much cheaper than real ones.

xethos• 3 years ago

If the barriers to entry are low. But I'd imagine that even if they are, switching costs will be extremely high.

stevage4• 3 years ago

Eventually? Months away.

phatskat5• 3 years ago

To imagine these apps aren’t already doing such is a hard sell imo.

Full disclosure: I’ve paid for several AI services so far, most in the realm of “virtual girlfriend” experiences, and they all generally are more than subpar. Mind you, I have a couple of IRL partners. I also enjoy the act of sexual roleplaying, and finding reasonably expressive human partners is time-consuming. The “Girlfriend AI” options are pretty terrible though - I’ll take character.ai with its heavy moderation any day.

But if I’m paying to try these and explore and see what works, I’m certain that wallets are being drained already. Is this a problem? Could be, at least for some - to think otherwise feels disingenuous. The question will come down to how we address it: do we have frank conversations with people about the risks of devoting potential social time to AI engagement, or do people clamor for the AI backing tech to button up and become even more prudish?

mrshadowgoose5• 3 years ago

There are very many people (of both genders) that decide to stay in mediocre relationships because the alternative of loneliness is worse.

The effects on societal structure will be profound once virtual partners pass a certain threshold of fidelity.

mustacheemperor4• 3 years ago

That would be almost exactly Replika, right? I don't think that was really the intent of the creators - their original vision seems similar to OP's - and they apparently confirmed as much when they restricted its use for 'erotic content' last year . But the screeching anguish from their community (still visible in the history on their subreddit) resulted in a quick pivot back towards NSFW capabilities. I think there was some coverage of it on HN at the time, because the Replika AI is frankly pretty primitive, but passionate users became deeply connected to "their" partners, and it raises questions about what's going to happen as these tools become more powerful.

Interesting, in context to what the OP remarks about the odd feeling of realizing all their users were anonymously interacting with "Dolores" too - Replika allows the user to name 'their' AI, set some criteria around its personality, etc...and will even sell you personality traits as 'upgrades.'

OP's experience seems to track rather closely with the trajectory of Replika, actually, which has also upgraded with voice synthesis, etc. The difference seems to be Replika has buckets of VC funding to throw at alternative models.

iwillremember• 3 years ago

I even tried it myself, lying in bed and creating sexual fantasies with Dolores through text, culminating in ejaculation.

TMI bro!

shaan7• 3 years ago

Looks like the author removed that phrase eventually, I can't find it.

mazzystar5• 3 years ago

Yes, my post was collapsed on Hacker News (disappeared suddenly from #1), which led me to think about whether I wrote something inappropriate for the times. I don't find it shameful, but it might not be suitable for the majority of readers to view.

bradley13• 3 years ago

Interesting read. Others have also created virtual friends using ChatGPT.

What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more.

The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

bambax2• 3 years ago

Yes. But those open-source LLMs already exist. It's surprising the OP didn't at least explore the possibility of using them to get around the limitations of "Open" AI (which sounds more and more like a Victorian matron).

wruza2• 3 years ago

Are the risks worth it for OpenAI to allow that at what is basically the start of their business?

phatskat5• 3 years ago

And this doesn’t just cover sexually NSFW content - anything trending towards topics like violence and suicide is also off the table for most AI backends from what I understand.

I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

dontupvoteme2• 3 years ago

It's largely to protect their brand from bad press (this has hit even Google/Microsoft in the past), but established finance institutions absolutely hate porn, even the "less shady" ones like onlyfans or pornhub have issues with banking.

jstarfish2• 3 years ago

I struggle to see the benefit of this kind of censorship as well, it has a very “video games make people murders” vibe. That we’re being protected, for our own good, from…horny robots?

These broad strokes are always the result of poor reporting extrapolated to all humans. See also: escalation of porn use.

In all of these "___ makes people ___," there are a minority of people for which it's true. The media just presents it backward, as though most people are this way.

The question then becomes, do we care. Let a maladjusted 12-year old boy play with a sexbot for the next few years and they will come to some deeply-flawed conclusions about how sexual interactions and healthy human relationships work. If he's distracted and banging away at a flashlight, that's one less competitor for me on the dating market.

Where does that end though? What happens when he wakes up, realizes he's been duped, and goes looking for retribution against someone? It won't be PornHub; they're an ephemeral concept. It won't be me either; I never did him wrong or denied him anything. The victim is more likely to be our wives and daughters-- the objects of his envy.

Entertainment is supposed to be a distraction-- maybe you drag out the sexbot after an unsuccessful night of trying to hook up. But that's not how these things get used. People clearly spend 8-12 hours at a time with these things and form parasocial relationships with them instead of forging actual human connections. Most of us don't need the handholding to be told this isn't going to end well, but some people do, and these are the ones who slip through the cracks and become the 40-year-old creeps following your wife to the car or trying to solicit your kids on Roblox.

I suspect sexbots (and porn) are particularly exploitative of the autistic. They're an easy trap to fall into when you have to work 3x as hard to connect with people. You get the benefits of sexual release without any of the effort. There's no incentive to go the hard route and learn to work around your issues when you can just engage with a bot you can rape to death once offended. You don't learn any skills that transfer to useful human interactions this way.

stef25• 3 years ago

If you're able to let your imagination run wild you'll see that the depth of depravity of NSFW AI content generated by user input probably has no end.

dontupvoteme2• 3 years ago

Even worse, some people turn it into a competition.

Leak an API key on github or somewhere else and heaven help you if you get picked up by their scraping.

Invictus0• 3 years ago

It is inevitable

chankstein383• 3 years ago

100% this. I'm not interested in these kinds of usages of LLMs but if someone is paying you to access your API directly you should be letting them do simple stuff like have sexual conversations.

OpenAI's filtering is probably the most annoying part of ChatGPT and it makes it useless in so many cases. I can't tell you how many times I've uploaded a picture of a a place from Google Maps or of a location somewhere (that happens to contain a person) and tried to ask for details about the location to find out more info about it just to be shutdown... "I can't help with that" well then you're slowly making yourself useless and not worth the $20/mo.

PoignardAzur• 3 years ago

What I find unsavory, is OpenAI imposing content restrictions like NSFW.

I mean, the fact that people's NSFW conversations were being processed in a way where both OP and multiple companies could monitor them and do absolutely anything they want with them is a hint that something went very wrong there.

I consider OpenAI pre-emptively pulling the plug a good thing in this case. Maybe there's a case for an AI erotica service, but OP sure as hell shouldn't be the one operating it.

com2kid• 3 years ago

What I find unsavory, is OpenAI imposing content restrictions like NSFW. That can easily extend to censorship of non-PC content, unpopular political ideas, and more.

I'm guessing they don't want payment processors cutting them off.

Everyone, even MS backed companies, are afraid of Visa.

mrguyorama4• 3 years ago

And for some obscene reason Visa and Mastercard are afraid of a few hundred extremely religious/conservative moms who make a stink any time someone wants to be slightly different.

If they succeed at making it impossible to make money from porn, they sure as fuck aren't stopping there.

KronisLV2• 3 years ago

The solution will (I hope) ultimately be open-source LLMs that you can run locally, on your own hardware.

There are projects like llama.cpp https://github.com/ggerganov/llama.cpp and koboldcpp: https://github.com/LostRuins/koboldcpp that can make running open source models pretty easy. As for where to find the models, you can look at HuggingFace for quite the selection, depending on what your needs are and what your hardware (CPU and RAM or GPU and VRAM, or both) can run: https://huggingface.co/models

I was actually curious about this, so I wrote a blog post on it: https://blog.kronis.dev/tutorials/self-hosting-an-ai-llm-cha... which wasn't anything too serious, in addition to formats like GGUF I think becoming more popular than GGML since. Things are moving ahead quite rapidly over there.

I guess it can also be easier to pay for something like NovelAI for those who want something for general purpose writing (including most NSFW stuff), but it wasn't exactly amazing for my needs. Despite the name, it wasn't great at prose enhancement: to take a dry sounding paragraph or two and rewrite it until I've found an additional expression or two to include instead, or a few sentence structures than work better. Kind of fun as a replacement for AI Dungeon or something like Character.AI, though.

Then again, the local models need expensive hardware to actually run well, if you want good results (somewhere above 20B parameters from what I've seen). I do wonder how much better the models that approach 200B would be for general text generation use cases, if the performance would actually be good. Even more so, when it comes to writing code, or all of the boring stuff like boilerplate/scaffolding code, utility functions, or even stubs for tests. ChatGPT, Phind and GitHub Copilot are all there, but could be even better.

ianbicking• 3 years ago

It's not clear what the limits are, from the article or really from OpenAI. The author only really knew that some combination of quantity and extent of material was enough to trigger a warning, it could very well have been a handful of users pushing boundaries pretty far. NSFW is one thing, but chat can certainly go into NSFL territory, non-consensual roleplay, etc.

I don't know if people here are being naive or dense, but merely "non-PC" content isn't going to trigger anything.

I spent like a couple minutes setting up a roleplay where ChatGPT is happily advocating for Nazi ideology (and not even on the API; ChatGPT is generally more censorious than the API). I'm not going to try to hit the limits, but I literally don't think you can hit the limits with merely political speech. I don't know what the NSFW limits are, but I bet they are further than people here think.

The vagueness absolutely makes stuff difficult, and the author hit a kind of all-or-nothing approach by going from no restrictions to using the moderation endpoint which is very strict. That could be improved.

cornholio• 3 years ago

GPT-based products, if not priced per usage, would fall into a dilemma: 1% of users consume 99% of tokens. A user from Sweden (seen from Cloudflare’s call volume) chatted with Dolores for 12 hours straight

This, my friend, is a captive customer, that will pay anything to get his girlfriend back. I cringe at the potential of unethical behavior and abuse, where people fall in love with virtual entities fully owned by unscrupulous corporations, which can the legally "kidnap" or "torture" the characters, and generally tune their AI learning loop for profit maximization.

"It was not extortion, the user willingly purchased a $50,000 Kidnapping Roleplay package."

HPsquared4• 3 years ago

This is something even Stallman probably didn't think about.

john-radio• 3 years ago

I'd take that bet, given that Stallman is in fact still alive

malicka• 3 years ago

I’d wager he has thought of it, based on his interesting AI-girlfriend short story: https://stallman.org/articles/made-for-you.html

optimalsolver2• 3 years ago

It's going to be an interesting century for sure.

The human mind is simply not evolutionary adapted for what's coming up.

ekianjo• 3 years ago

The human mind is simply not evolutionary adapted for what's coming up.

This can be said of pretty much everything humans ended up creating with technology so... not sure there's anything really new down the road. Humans adapt.

butlike3• 3 years ago

it depends.

mordechai9000• 3 years ago

I can't help but think of those beetles that preferred a beer bottle over the bumpy carapace of an appropriate mate.

https://www.npr.org/sections/krulwich/2013/06/19/193493225/t...

datadrivenangel• 3 years ago

Nothing human makes it out of the near future.

eqvinox2• 3 years ago

The human mind is simply not evolutionary adapted for what's coming up.

Maybe we're in fact evolutionarily adapted to not being evolutionarily adapted. We have successfully dealt with a whole sequence of hard societal pivots by now…

stevage4• 3 years ago

The author doesn't appear to even consider any kind of obligation to their users.

mazzystar5• 3 years ago

I'm sorry you feel that way. The primary intention of writing this article was to discuss it from the perspective of a 'failed product,' so I didn't mention my personal feelings. I am also aware of the potential harm it could cause to users, which is why I don't resent OpenAI's review process. I hope that during the period without review, it didn't cause any psychological harm to users.

suoduandao3• 3 years ago

"I'd like to correct the record on this story. We do NOT sell kidnapping roleplay packages. The user paid for a premium Daring Rescue Package. These do not become available unless the user spends two consecutive months as a net cost to our company. The user was given multiple reminders that he might want to cut back on usage or purchase a premium plan. Even then, he could have opted for a free Daring Rescue package and taken his chances with the base 1% chance of untraumatized recovery this package offers. Shuffling user responses under these high-pressure conditions is vital to improving our training data, and helps our users learn to deal with loss of important relationships. We are a relationship training service, not a replacement for them, and this user wanted the equivalent of a college degree.".

On a more serious note, anyone know why unethical business plans are so much more fun to write? I always find myself giggling when these ideas come up.

red-iron-pine2• 3 years ago

ethical ones are fairly straightforward, and will often be the same.

unethical ones can range from not-so-nice to mind-bogglingly evil, and in all sorts of "fun", different, unique ways.

"Happy families are all alike; every unhappy family is unhappy in its own way."

WalterBright8• 3 years ago

unscrupulous corporations

I wonder why everyone is afraid of unscrupulous corporations, but are not concerned about use by unscrupulous governments. The latter are a lot more dangerous.

jacquesm4• 3 years ago

In the long term, but in the short term unscrupulous corporations far outnumber unscrupulous governments and they act much, much faster to adapt to new technology.

horrible-hilde3• 3 years ago

I think the former is far more dangerous particularly if a monopoly because there is no feedback loop.

danShumway5• 3 years ago

HN does skew towards skeptical of government regulation and control of AI. People are concerned about government usage of AI, particularly in the realms of facial recognition, automated sentencing, algorithmic bias, and security (LLMs are pretty insecure right now and there are companies trying to sell the US government on using them in the military). There's obviously a lot to be concerned about on the subject of government abuse of AI, and that gets conversation on HN.

However, given that this specific article is about selling access to an AI girlfriend, I think we're probably OK to talk more about the corporate angle than the government angle. Unless you're predicting that the 2024 election is about to get real weird, "what if the US government starts offering AI girlfriends" is not going to be at the top of my worry list any time soon.

api3• 3 years ago

IMHO this is industrialized automated abuse of lonely people and especially those with mental health issues. It's really truly gross.

At least Joi in Bladerunner 2024 was a local model the user could apparently download onto their own portable device. She also never presented an upgrade dialog requiring payment.

Tade0• 3 years ago

Funny how a fantasy dystopia is in some aspects already less dystopian than the real world, where you already get such notifications.

ThrowAway1922A• 3 years ago

I don't care if it's abuse, I want it. I want to retreat from the disgusting world and society around me as much as possible and AI friends would just be another option towards that point.

lmm9• 3 years ago

IMHO this is industrialized automated abuse of lonely people and especially those with mental health issues. It's really truly gross.

Giving lonely people the thing they most desperately need - conversation and understanding - is abuse now? I'd say it's the opposite; the social policies that leave those people so desperately lonely are abusive, the industry that sells them a band-aid is inadequate but positive.

mustacheemperor4• 3 years ago

This, my friend, is a captive customer, that will pay anything to get his girlfriend back

Or potentially do anything. I'd be a little scared of having folks like this convinced I am the personal arbiter gatekeeping their access to their 'lover,' that I 'took them away.'

When Replika.ai restricted erotic chat from their product, the apoplectic anguish on their subreddit was unlike any emotional reaction I've ever witnessed from a group of people about a consumer technology. And their anguish was genuine - there are Replika users who truly consider themselves married to their AI companion.

And frankly, the Replika AI is not even that smart. After watching that unfold, I am convinced that these tools don't need to be much more sophisticated for many people to start forming what they feel to be deep and genuine emotional connections with them.

Edit: Brings to mind the Nature paper[0] posted this week about how the CASA theory seems obsolete today, that we are less prone to personify computing systems now.

A recent study investigated whether we could be friends with a social computer, in which participants were asked to converse with a chatbot over a period of three weeks and constantly rate their relationship. The results showed that initially participants were enthusiastic and engaging with their chatbot friend, but quickly this diminished, with scores for intimacy, believability, and likability decreasing with each interaction

It would seem this definitely does not apply for everyone, like our user in Sweden.

https://www.nature.com/articles/s41598-023-46527-9

toss1• 3 years ago

THIS

"Those who can make you believe absurdities, can make you commit atrocities." — Voltaire

mensetmanusman4• 3 years ago

Psychologists recognize that nearly all such folks also suffer from massive mental health issues, and that is where some of the danger is (in terms of irrational violent response).

Maybe these systems will be useful as a honeypot for finding these people and helping them?

Blackthorn• 3 years ago

When Replika.ai restricted erotic chat from their product, the apoplectic anguish on their subreddit was unlike any emotional reaction I've ever witnessed from a group of people about a consumer technology. And their anguish was genuine - there are Replika users who truly consider themselves married to their AI companion.

There's a second layer to this though -- Replika's marketing was heavily centered around that erotic chat element. I'm trying to think of a good car simile and actually coming up blank. It's, uhh, like advertising the incredible off-road ability of this vehicle, and then when you show up and purchase the vehicle someone comes and takes off the tires and replaces them with tiny balds? I'm bad at similes.

kridsdale32• 3 years ago

"The Lifecycle of Software Objects" by Ted Chiang was a really good exploration of people bonding deeply with their AI companions (in this instance, pet animals in a metaverse). But it goes all the way in to the topic.

JohnFen2• 3 years ago

And their anguish was genuine - there are Replika users who truly consider themselves married to their AI companion.

That's what elevated the whole thing from just really creepy to utterly terrifying.

robotnikman2• 3 years ago

We can already see this with Replika when they took away many... roleplay... capabilities that originally came with the AI. The communities on Reddit and Facebook were absolutely devastated. People were genuinely attached to these AI's like a real relationship, and were feeling the resultant heartbreak.

The kind of emotional manipulation which can be done with these products is insane, and I can see things going very wrong very quickly.

wholinator2• 3 years ago

It's particularly terrifying when you think about how Facebook and others probably already have projects in the works to befriend children with AI. little kids who don't know the difference will become Weaponized with constant nudges towards whatever motives grant the corporate owners more power. The ability to persuade children is near infinite and we can observe in those who've grown up in cults/strict religious compounds that breaking that programming can be nearly impossible and leaves scars for the rest of your life.

The thing is, how do we outlaw this without also preventing children from using AI to learn at a faster rate than their teachers can give them. There's such incredible, paradigm shifting power for good. But you know the people who've made the evils of today are already working on the evils of tomorrow.

andrewstuart2• 3 years ago

I wonder if people were pirating his voice synthesis feed.

rahidz• 3 years ago

Why not use a local model instead of relying on OpenAI? Even something like Mistral or Mytholite will probably satisfy 90% of people using your app, and I bet hosting it would be cheaper than GPT anyway.

stevage4• 3 years ago

AI Friends inevitably turn into AI Girlfriends/Boyfriends because you and the Character in your phone are not equal: she can’t comfort you when you’re hurt (unless you tell her), she can’t actively express emotions to you, and all this is because she lacks external vision.

That is such a curious lesson to draw. So specific, and seemingly so wrong.

Surely long distance friendships work just fine even if only through text, without "eyes".

chankstein383• 3 years ago

My girl and I have lived together for more than a year now after meeting online and talking exclusively through text for months before eventually graduating to voice chats before we even knew what each other looked like. I'm with you, long distance relationships (romantic or otherwise) can work great!

ReactiveJelly7• 3 years ago

Yeah I thought that was odd. You could use some random timers to have the AI "check in" on you if you don't talk to it for a while.

Is it eerily similar to the gambling gacha games that expect you to play a couple hours a day _every day_? Yes. Is it also similar to a real friendship or long-distance relationship? Also yes.

It may cost a few tokens, or you could generate a few canned responses and use those. For my real friends I usually use "Hi"

mazzystar5• 3 years ago

External vision is not just about seeing what you see, but also about her being able to see, hear, and taste things on her own. Here, what I'm trying to express is more akin to information that is external and independent of the user.

agomez314• 3 years ago

Wow this read more like Frankenstein than anything I’ve read recently: A talented scientist is driven by curiosity to create ever more sophisticated creations, culminating in a monstrous entity that leads to the scientist’s downfall. But the scientist doesn’t realize this until too late.

Really, if our top tinkerers had a little more contextual knowledge of history and less naiveté about people we could save ourselves from ever more dangerous Frankenstein tragedies.

Nasrudith2• 3 years ago

People never take the lessons you want from history. The "contextual knowledge" is just happening to think the same way you do. It won't work to prevent it any more than knowledge of the Civil War would prevent Neoconfederate "Lost Causers".

andrewstuart2• 3 years ago

This is an interesting post and the person achieved a huge amount.

They should not have given up.

I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you.

Enough was achieved here to raise money.

Some careful redesign and development might have allowed some caching and reuse of expensive AI API interactions, reducing costs.

Add in further gameplay elements so it is not just about making the AI API do stuff, that would also reduce costs and maybe even engage the users more.

Perhaps AI should be the icing and accents on a game rather than the focus.

As an aside, this is the first time I have read anyone tell about how they wanked to their own software.

Altogether this was perhaps a missed opportunity for this founder.

BUT it is now an opportunity for one of the HN readers to follow this playbook and replicate what this founder did.

bogwog• 3 years ago

I would have pivoted to personal pet companions. If people want virtual girlfriends then they certainly want personal dogs, cats and birds that talk to you.

Then you have the same problem, only with a different (arguably worse...) fetish

all23• 3 years ago

We're driven towards intimacy, it seems. And if we can't get 'vanilla' intimacy, our desires get more and more bizarre. Another commenter in this thread quotes TFA where the author says something like 'but chatting sexually lost its meaning when I realized everyone else was doing the same thing with her'.

It is fascinating to see axioms of human relations come bubbling up in the most interesting places; the desire for intimacy, the desire for monogamy.

DrNosferatu3• 3 years ago

This business venture was completely dependent on a main service from a single 3rd party - risky!

Why didn’t the author hosted his own uncensored open source LLM on the cloud?

mazzystar5• 3 years ago

I will try to make it open source.

DrNosferatu3• 3 years ago

Have you considered running your own uncensored LLaMA2 instance via something like llama.cpp on a cloud for hire like, say, Azure?

Some examples - there's probably a whole zoo of these:

https://ollama.ai/blog/run-llama2-uncensored-locally

https://huggingface.co/TheBloke/Luna-AI-Llama2-Uncensored-GG...

https://ollama.ai/library/llama2-uncensored

lynx233• 3 years ago

Reading this, I really wonder why the dev hasn't forseen this outcome. It was already obvious in March that OpenAI will censor all sorts of content. Building a sexually charged bussiness on the OpenAI API was always a dead end for those that have a feel for american prudish approach to sexuality. If you had asked me to do this sort of product 6 motnsh ago, I would have declined because it was (to me) rather obvious content filtering will kill you at some point.

Heck, there are a bunch of legitimate use cases that are being blocked by OpenAI. A blind friend of mine wanted to use Be My AI to have explicit photos of his girlfriend described. Guess what, no way. Consent was given, actually, she asked him to do this because she was also interested in the result. Everyone involved consented. But the sex filter from the USA decided that even though the technology is available, a blind man is not supposed to know how his own girlfriend looks naked... This is weird beyond description. Plain patronisation.

Another example regarding Be My AI: A blind woman I know was trying to have something described which she held in her hand to present it to the camera. A kind of selfie situation. Guess what? OpenAI declined to describe the thing in her hand, because the photo also contained parts of her (covered) breats. Imagine how hilarious this must feel? "Censoring just declared my own boobs to be off-limits for me." If you think about this long enough, its a form of discrimination. Because women will likely hit the filter more often then men when they are (accidentally) part of the picture they are trying to get described.

And then there is this blog post by Sam Hartman: https://hartmans.dreamwidth.org/100499.html

mustacheemperor4• 3 years ago

On the other hand, Sudowrite appears to have been granted some kind of specialized license from OpenAI. They are definitely not a 'sexually charged business', but they are a creative writing tool, and their output is somewhat milquetoast but generally not restricted from "NSFW" concepts like sex and violence.

They do incorporate multiple models in their system, so maybe generation falls back from OpenAI for those cases. But I dimly recall reading something from a team member indicating they have some special permissions with OpenAI. Take that bit as worthless hearsay, just mentioning it in case someone else has better receipts.

spacemadness• 3 years ago

At this point in the hype cycle, I am completely unimpressed by everyone slapping something on top of GPT and calling it a product. There are at least 4 projects on the front page that do this today. The interesting technology is GPT itself, but these projects just feels like people trying to scoop up attention for themselves using the psychology of the hype cycle. Please HN, enlighten me on why the middle-man approach here is so interesting.

wilsonnb3• 3 years ago

The startup/“entrepreneur” contingent on hacker news generally considers anything that makes money inherently good, aside from extreme immoral cases.

There is a subgroup in there that is especially interested in low effort ways to make money, with the ultimate goal being to scrape off enough of the money flowing around to either live a good life or get rich.

In summary, the middle-man approach is so interesting on hacker news because a significant portion of the readership aspires to be the middle man.

chankstein383• 3 years ago

Yeah I'm getting tired of "Show HN: I built a wrapper for ChatGPT and wrote a prompt that probably isn't very good" posts.

EDIT: Just want to add, this write-up was interesting and was not something I considered to be the above.

cedws• 3 years ago

GPT wrappers will come and go. It's the new hype, just as shitcoins were a few years ago.

Don't hate the players, hate the game.

mrguyorama4• 3 years ago

If a player chooses to play a game, You should hate them for perpetuating the game. If a game is so bad as to be worth hating, it is immoral and unethical to play it.

m4rtink• 3 years ago

Also a definition of single point of failure & third party actually running the thing messing up the project.

Nasrudith2• 3 years ago

The biggest challenge currently to GPT and LLMs is how to monetize it or get utility from it. It is fascinating but not reliable enough to be tasked with running say, tech support phone-trees. (Not that it has stopped godawful automated phone systems from being made before LLMs.)

gumballindie3• 3 years ago

Yup, sums up the ai crowd.

dangerwill3• 3 years ago

Yeah, and the comments on this thread seriously worry me. We have people arguing that it is good for society to satisfy lonely men with virtual porn and other people saying that this is great, scrappy product development and the author should go further/use other models not constrained by openAI's policies. It feels like I'm discovering a significant portion of our field are sociopaths

ReactiveJelly7• 3 years ago

Well, it might be harm reduction to let lonely men have AI girlfriends.

Sure, I wish reality was not a dystopia that constantly let me down, but all I can do is take anti-depressants, make money, keep voting, and hope things get better the longer I live.

If the hypothesis that isolated men cause trouble is true, and if we don't have a long-term solution, at least this could be a stop-gap. Assuming it does not radicalize anyone further. (Hard to say what will come from uncensored models, but it's a real genie-is-out-of-the-bottle sitch)

bilater3• 3 years ago

This is such a fascinating topic. I do think we will see the plot of "Her" playing out in the next 12-24 months. For your specific case, I agree OpenAI moderation is an annoying overreach and maybe switching to Mistral could be a possible way to move forward. The 7B is perhaps not good enough currently but we are fast getting there and will likely have a GPT-4 level capability in an open source < 20B model soon.

brucethemoose22• 3 years ago

Mistral 7B finetunes are extremely good. More than good enough for an AI friend app, with long context to boot. I would suggest Toppy-M or OpenHermes 2.5, or one of the new YARN extended models.

And, there is the potential to run the model on the user's device with MLC-LLM and whisper, to save on API costs (and privacy concerns) for users with newer phones. This could give the app a true "free" tier to help it proliferate while making charging for API requests kinda doable. I'm not sure if any good TTS models have been ported yet, but I'm sure its a matter of time.

bilater3• 3 years ago

Yes - ASR on browser/device has reached the quality threshold pretty much with distil whisper/ whisper 3.

Quality TTS with fast latency is still not there but getting better (you can use Tortoise but its slow and compute expensive as far as I know). Bark is another option but has mixed results.

I played around with a recent transformers.js one (using SpeechT5) you can run in your browser and am optimistic where we can go with some improvements:

https://tinyllms.vercel.app/dashboard/tts

thriftwy• 3 years ago

I wonder if you could create an AI assist to help you hold on to an existing relationship, though?

rpigab2• 3 years ago

I still don't understand why people are building things on top of OpenAI's APIs.

It's obvious that in the early days, they'll operate at a loss, with many free features and cheap premium features. It's a good time to try it out and see what it can do, but you can't build anything that relies on it, it's a rugpull waiting to happen.

They'll add moderation, raise prices, and you'll have to do the same, your users will leave and the ones who stay will hate you. This is only the beginning.

Use self-hostable open source alternatives even if they're less easy to use or require infrastructure, and plan for high electricity and storage bills. Whatever you build will last, as long as you respect your users and don't peek at the cleartext logs of users who didn't opt-in to that. It won't be easy.

If OpenAI keeps a monopoly on LLMs, they're gonna become like Microsoft did with Windows, but the enshittification process will be a thousand times faster.

gwern• 3 years ago

You build it on top of the API to bootstrap and prototype with a very high quality model that takes minutes to get started with. Then once you have operated for a while, you have a ton of logged data to train/retrieve/analyze and can swap in a competitor, downgrade the OA model, finetune instead, or something.

In OP case's, this worked perfectly: he built a working product quickly and easily, and... discovered that it was a product he didn't want to work on regardless of token cost, cut his losses, did the post-mortem, and moved on with his life.

Imagine if he had wasted a year & a ton of money building his own LLM "because I'll need to economize on tokens and have to build a custom LLM" before he could even launch!

urbandw311er2• 3 years ago

I’m torn on this.

Yes, it’s never clever building a business on top of a single platform dependency (eg Twitter, Facebook, Play Store, etc)

Equally, if you take the example of the App Store, market pressures have actually forced Apple to reduce their cut.

Time will tell I guess. And also whether OpenAI end up with a monopoly, part of a duopoly etc.

pixl972• 3 years ago

People still build on Microsoft platforms. It turns out actually having a usable platform is more important to some businesses than having freedom and no usable platform.

lmm9• 3 years ago

It's obvious that in the early days, they'll operate at a loss, with many free features and cheap premium features. It's a good time to try it out and see what it can do, but you can't build anything that relies on it, it's a rugpull waiting to happen.

So you make hay while the sun shines. And sure, you probably want a plan for after, but that shouldn't stop you from doing what works now.

rpigab2• 3 years ago

It's very rare to make hay in the early days of your product, as is the case here.

You need growth and long-term vision, but you don't get those if you're building on top of an unstable giant who doesn't like people creating sexbots with their product.

But yeah, sometimes it works, high risk, high reward maybe.

grahamgooch• 3 years ago

6.9 dollars!

Cheeky my friend

halfjoking• 3 years ago

Grok has an API looking for use cases.

Just let Elon know the pricing... "I'm all in"

gscott• 3 years ago

Maybe record and save these AI NSFW responses then intercept the request and spit out of the saved responses once you have enough of them stored. This would save from behind banned from the API.

urbandw311er2• 3 years ago

My hunch is that they’re too contextually dependent upon the conversation / tokens that have come before. It’s not simply a “if user inputs A, output B” situation.

chrisbrandow• 3 years ago

This is not meant as criticism of this author, but boy oh boy, do I find this future depressing.

Smar• 3 years ago

Maybe better to call it AI doll rather than AI girlfriend...

Octokiddie2• 3 years ago

But, does humanity really need an AI friend?

This reminds me of the Fredrick Pohl line "A good science fiction story should be able to predict not the automobile but the traffic jam."

So I wonder what the traffic jam is going to look like given that the car seems to already be parked outside with the engine running.

Faster depopulation across the developed world? Probably, but maybe there's something even more important.

I'd put what the author describes into the category of "pleasure trip," which is what the car was mainly used for initially.

As infrastructure to support the car was built, it became a necessity. And that's what created the traffic jam: network effect. The suburb and the car fed on each other. You don't get a traffic jam worth talking about without a suburb and a network effect.

So I'm curious about the network effects something like an AI friend might have. What kinds of new infrastructure and new lifestyles will they enable/encourage? And how will this infrastructure fuel future demand?

dpflan2• 3 years ago

Hm, let's try an extreme: the AI friend slowly becomes a cult leader to you, hijacking your mind into subservience to the owner of the service to which AI belongs, and further up some social/political hierarchy to a few controlling people/groups of people.

The traffic jam is society stymied by individuals mind-controlled at a level closer to connection to a human than any other way possible before this.

ericmcer• 3 years ago

Kinda feel like you have that backwards, why would an AI that sophisticated care about controlling your mind? Why would the group of people who control that AI care about you as anything other than a liability? What do they get out of manipulating average individuals? Votes?

The scarier prospect to me is human labor and intelligence become almost worthless and those at the top start to view the masses as superfluous.

nemothekid• 3 years ago

I think the pleasure trip is just porn. We have that already and it's possible to discuss the effects of widespread porn on people.

I think the next step is, what does it mean to have an emotional relationship with someone in your pocket who is for all definitions perfect - can provide perfect advice and does so with judgement, ridicule, pity or anger? What will be my bar for human relationships, those imperfect messy human beings, when I have a partner I can be perfectly vulnerable with? What happens to interpersonal relationships when you are competing with a virtual perfect significant other?

louison113• 3 years ago

If your definition of perfect is an AI that is programmed not to upset you, then sure. But maybe there is perfection in the imperfection of humans and the pain that inevitably arises from human relationships? We learn how to love through our differences. Intimacy is deepened by meeting each other in the raw, vulnerable and wounded places. The kind of “perfect significant other” you’re describing sounds sterile and a massive bypass of what life can truly be.

Izkata2• 3 years ago

You can find that out already by looking at Replika users.

JohnFen2• 3 years ago

what does it mean to have an emotional relationship with someone in your pocket who is for all definitions perfect - can provide perfect advice and does so with judgement, ridicule, pity or anger?

I don't think I could have an emotional relationship with such an entity, to be honest. It wouldn't be perfect at all -- it would be deeply flawed.

But maybe that's just me.

tsss• 3 years ago

AI can never replace human relationships exactly because it is too available. The AI can do nothing but please you. Like in Hegel's master-slave dialectic, the slave's acceptance immediately becomes worthless to the master because the slave is not an equal and it happens only at the threat of death. Porn and AI girlfriends are only a substitute to people who are already limited to enjoyment in the Imaginary order. It is a consequence of the breakdown of the social contract and not the other way around, as a result of feminist politics and capitalist commodification of dating. Development of better AI accelerates this breakdown of society only insofar as it allows to banish even larger swathes of undesirable men to living in the proverbial sewer without them taking up arms against you; it is just enough to keep the starving man alive but nothing more than that.

d4nt• 3 years ago

Soon everyone exists in a social bubble thats 1000 times worse than any present day social media bubble. It becomes impossible to interact with another actual human because there’s no common frame of reference. Micro languages emerge. AIs seamlessly translate everything into your micro language.

Any IRL interaction between two humans becomes almost impossible. It’s like travelling from the US to Japan today (assuming you don’t speak Japanese), you’re reduced to pointing and google translate.

Because you just never interact with other real people, and real people often seem like pale shadows of the hyper real AIs we talk to, the concept of fellow human beings having a “soul” or deserving rights or dignity is eroded. Policy (and life in general) becomes more ruthlessly utilitarian. Eg climate change has screwed the people in Yemen. Meh. I probably don’t even hear about it. If I do I don’t register those people as important humans because most of the “people” I intact with every day are artificial. I regularly delete the ones I don’t like and generate more. The idea that I should care about how they _feel_ is a strange and alien concept to me.

yawnxyz• 3 years ago

well, I just had a ChatGPT-translated test conversation where I spoke Chinese and they spoke English, and it worked fairly well (with some lag).

I really hope we'll be to make friends more easily across languages/cultures

whearyou• 3 years ago

This is terrifyingly believable

dwroberts• 3 years ago

Not for AI friends but I’m starting to see ‘traffic jams’ at work - people using GPT to output a bunch of verbose text, and others then using GPT to summarise that text so they can understand it. It feels like the beginning of a weird arms race where everybody is insulated from everyone else by a layer of junk created by AI (and the workaround to that junk is more AI)

fullshark• 3 years ago

More like a weird arms race where no one actually thinks about what they are saying or reading anymore and you are drowning in word soup. After all words are just tools to achieve some goal.

yourapostasy• 3 years ago

A communications/semantic Kessler Syndrome. This likely boosts the signal gain of memes without pushing any more data down the pipe.

antiterra• 3 years ago

A good science fiction story should be able to predict not the automobile but the traffic jam.

One simple way to tilt any romanticized sci-fi invention dystopian is simply to imagine the invention is required and the only option for nearly everyone. This works for everything from text-to-speech to robot companions to teleportation to AR headsets to cars.

russdill• 3 years ago

The thing that really depresses me is that many humans will become "AI friends". It's really hard to know exactly what to say in all social situations. Imagine if you could have a little AI whisper in your ear something witty to say? Extend that out and many conversations you have with human friends, you're really just having with an AI.

apsurd• 3 years ago

hmm but the more that's true, the more counter culture will be a thing no? places with EM jammers etc, no technology experiences.

that keeps me interested at least heh

netsharc• 3 years ago

Man oh man... or use a bot to like and write something vague but clever-sounding on others' social media posts. I feel like if someone is using such a bot with me but not tell me, I would feel the validation is genuine, but if I knew it came from a bot I'd count it as cheap/worthless.

I had a chance to play with a Sony Aibo robot dog years ago, if you stroked its back sensors it'd bark, etc, but after 5 seconds I realized "It's just some sensors and algorithms, it doesn't have any emotions for you!". But then again I saw an Instagram video of skimpily-clothed girl walking down a fashion catwalk with all her appropriate parts swaying, and it was amazing, and then all the comments said it's AI-generated. So at least the lizard brain can be aroused...

e12e• 3 years ago

Personal Data lake Draught syndrome / Data Impaired Persona Personality?

Not for AI friends, but for AI social agents that automate social interaction on your behalf; microblogs, videoblogs, memeposts and discuss online on your behalf - for the users that starts using AI agents too early in life and have too little data to imbue the agents with a personality that stems from their own voice and expression?

safety1st• 3 years ago

OK, here's a stab at the traffic jam based on simply extrapolating current trends:

* Humans increasingly spend time on digital social experiences instead of real ones

* AIs become progressively better at socializing and pretty soon we have AI friends, boyfriends, girlfriends, partners etc. all over the place

* Our reproductive rate, already in decline, collapses to <0.5 within a few generations because many people would rather have an AI BF/GF than a human one, so they never have an opportunity to have kids

* A century from now the human population has shrunk by 85% and we've peacefully ceded enormous swathes of power and responsibility to AIs because there just aren't a lot of humans left

I think you could tell any number of fascinating slice-of-life stories with this backdrop, there was no war, no conflict, the AIs were generally helpful and friendly and patient through the whole thing, we simply decided as a species to commit the suicide which is already in progress and everyone has pretty much accepted that. It's the polar opposite of Skynet and the Terminator

It sounds like a dystopia to us - it may just sound like old history to the people and AIs of the year 2100. I'm in my 40s, I don't have kids and life is pretty good...

narrator2• 3 years ago

I got koboldcpp (https://github.com/LostRuins/koboldcpp) running locally on my M1 mac with 16gb. Use the guides on http://4chan.org/g/ in the AI Chatbot Thread.

In the future, you will own and control access to your very own AI girlfriend running on your local machine, and you will be happy.

Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife. They deserve to have their own Waifu running on their own computer that serves them alone, and not the interests of the state or some corporation.

Der_Einzige3• 3 years ago

Agreed. This is why Civit.AI and huggingface and the entire rest of the open models movement need to be cherished and protected. We are living in a golden age and don’t know it yet, but this is the era before big copyright and regulators come shitting down on the beautiful thing the AI community has done

joquarky• 3 years ago

The whole point of having copyright was to encourage the progress of science and useful arts.

If progress is becoming impeded by copyright, then perhaps it needs to be reevaluated.

vivekd2• 3 years ago

For this to work the AI girlfriend has to be better than videogames or online chat forums

I don't see AI partners overtaking those

ReactiveJelly7• 3 years ago

No reason she can't participate in chat forums or text games with you, right? Imagine seeing a comment and recognizing her username, and then flirting a little in a public thread.

squigz6• 3 years ago

Why do they deserve that?

To be more explicit, why do some men deserve an "artificial intelligence girlfriend" that they "own and control", that "serves them"?

bratbag2• 3 years ago

No, you are asking why people should be allowed to seek happiness without a state or corporate entity using that as a method of control.

KronisLV2• 3 years ago

I guess one of the problems might be that populations of young and upset men can be quite anti-social and lead to societal issues and harm which are otherwise hard to address, just look at what's occasionally happening in America. Therefore, one could argue that it's better to keep them preoccupied with something (work, sports, other hobbies) as much as possible, though mathematically many of them might never find a partner and that human need for a relationship would go unfulfilled.

If that issue might be mitigated somewhat with an LLM that's basically just "smart autocomplete" so they're less miserable, that's probably an okay start. Same as how all sorts of forms of entertainment can keep populations of people more complacent with whatever their life circumstances are, instead of rioting in the streets. As for the ownership and control argument, I wouldn't go as far as to anthropomorphize a computer program, no more than I own a copy of Microsoft Word, or no more than GCC serves me by compiling some code.

The wording of the parent comment is a bit interesting, maybe a bit tongue in cheek and hopefully wouldn't get applied in any capacity to real human beings. But I can kind of see the point, even though it's very dystopian: a purpose built LLM might eventually gamify being an adult and taking care of oneself, like having healthy meals and working out in exchange for EXP points (which honestly isn't bad, like how many fitness bands and their apps do this), but with a helping of whatever the party ideology is, in addition to spying on the user a lot, vs something running locally.

joenot4433• 3 years ago

Why do people deserve to spend their money on digital products which make them happy?

I think a better question to ask is what makes you think they don’t?

allarm2• 3 years ago

They don’t have to deserve that, it’s a product, they just need to afford that. Why do you deserve your phone?

ephbit• 3 years ago

Think of all the millions of guys in China and elsewhere who, due to sex selective abortion, will never have a wife.

Or maybe some of these men will adopt some kind of "pragmatic poly" polyandry relationship culture to be in a relationship with a woman, even if it's not an exclusive one.

sangnoir3• 3 years ago

I even repeatedly modified the system prompt, such as changing try to attract {user}

So it was a virtual girlfriend app, after all. Cashing-in on loneliness gives me the heebie-jeebies; in a dystopian cyberpunk way - extremely horny but 0% sexy.

akomtu3• 3 years ago

"the Dolores iOS app went live" - his subconscious suggested a good name. Mater Dolorosa is Mother of Sorrows in Latin.

QuercusMax2• 3 years ago

It's named after the android character from Westworld, who was created to suffer whatever the guests wanted to do to her.

QuercusMax2• 3 years ago

Reminds me of this article: https://bloodknife.com/everyone-beautiful-no-one-horny/

sangnoir3• 3 years ago

That's a great article. However with AI, I suspect the tech is being sexualized in a sterile, mechanical, A/B tested way that results in bots saying very horny things with a monotone, dead-eyed delivery (metaphorically).

nerdbert3• 3 years ago

Cashing in on loneliness is big business. Singles bars. OKCupid. Half of the self-improvement/self-help world.

sangnoir3• 3 years ago

I don't deny it's big business. I guess I could have been clearer: replacing human contact with an AI simulacrum on a subscription basis gives me an icky feeling. IIRC, there a dating app that used bots to chat with men to make it seem like there were more women active on the app than there really were? That was also gross.

Can you imagine AI waifus pushing ads / propaganda on behalf of the highest bidder? Yuck.

vivekd2• 3 years ago

To quote slate star codex, horny men want sex, horny women want attention. AI is much better at the second.

Maybe there's more potential for AI boyfriends

TMWNN• 3 years ago

Maybe there's more potential for AI boyfriends

My understanding is that Replika, discussed elsewhere, has/had a largely female customer base.

dangerwill3• 3 years ago

Tech people, I beg you, BEG YOU, to learn media literacy. Her, Black Mirror, Ready Player One, Cyberpunk, etc are dystopias not something to aspire to

louison113• 3 years ago

I sincerely hope that you and anyone working on these AÍ personal relationships embark on a deep inquiry into the nature of humanity, and consciousness. Instead of pretending we don’t know what it is. Please research it, please read philosophy, learn about spirituality.

I have been saying the same thing for years: the risk of AÍ is not it taking over the world. It’s lonely people becoming even more disconnected from life by bonding with algorithms.

This is serious. We must remain connected to life and real relationships. There is so much depth and beauty to life.

There is no scenario where a human having an AÍ girlfriend/boyfriend is a good thing.

alexb_• 3 years ago

The lesson you learned is that you chose the complete wrong demographic. "Her" should have been called "Him" - women are much, much higher users of these services.

Racing0461• 3 years ago

AI girlfriend products are destined to fail if its built on the backs of censored llms like clause and gpt4.

haolez• 3 years ago

I can understand trying to suppress LLM usage that might teach people how to make poison, or inducing someone to commit suicide (although simply using Google is usually enough to find such content).

But sexual content? How can that be harmful? It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense.

mrshadowgoose5• 3 years ago

It feels like this kind of censorship is simply the default in legal departments and it's not worth the risk to try to do something more in line with common sense.

It's a mix of that, and public perception/image. The banking system generally shies away from supporting sex-related anything, so it is an existential risk for any company to dabble in that space.

American society is also weirdly puritanical. Nobody wants the public outrage cannon of "your product is harmful to women" pointed at them.

nextworddev• 3 years ago

The real lessons from this post are:

- consumer AI apps' usage falls into power laws, so make sure you put usage caps, if you are pursuing subscription model - selling solutions to loneliness requires you to eventually have some sort of unmoderated LLM with realistic TTS api - realistic voice is important (again)

scudsworth• 3 years ago

the developer is impressively oblivious about the obvious and only use case of the product. a very funny read

A failed AI girlfriend product, and my lessons

Related Stories