Artist feeds childhood diary into GPT-3 to have a chat with herself

https://nwn.blogs.com/nwn/2022/11/michelle-huang-ai-art-gpt-diary-conversation.html

by SLHamlet • 4 years ago

162 97 4 years ago

nwn.blogs.com

powera• 4 years ago

The Twitter thread this is based off of <https://twitter.com/michellehuang42/status/15970054894137139...> has been submitted 8 times in the past 2 days, but never made the front page.

https://news.ycombinator.com/from?site=twitter.com/michelleh...

biggerChris• 4 years ago

That’s the issue with this site.

Tao33002• 4 years ago

I'm trying not to go there if I don't have to, and to mostly refer to it in the past tense.

joshe• 4 years ago

A new reason to write more or at least record more of yourself so that you can create things like this. Save your emails!

andreyk• 4 years ago

For some reason this article does not have the actual chat (just a short excerpt)... You can see more of it here https://twitter.com/michellehuang42/status/15970060064036208... and here https://twitter.com/michellehuang42/status/15970060108202106...

It's fun, but not so different from a conversation you could have with a psychic who is good at cold reading. It'd be more interesting to see what would have happened if GPT-3 was actually fine-tuned with all the journal entries, I think.

I do agree a lot with this conclusion: "This is the stuff I think that has the most interesting ramifications: more broadly, more immersive human / computer interface loops, from conversation with virtual therapists to in-game interactions for virtual worlds, given there is user input, AI could be used to train highly customizable responses or generate unique storylines per use."

visarga3• 4 years ago

Not just GPT-3, but Stable Diffusion is also therapeutic.

b800h2• 4 years ago

I feel a bit cheated by this. She fed in only 13,000 characters because more would be expensive. Fair play, you're an artist so it's all about the concept but the lack of follow-through here is disappointing.

olddustytrail• 4 years ago

Why feel cheated when you can feel opportunity?

This is your chance to become a patron of the arts! Pay to follow the concept through and realise the vision.

b800h2• 4 years ago

Yeah, fair point!

bikingbismuth• 4 years ago

I feel fortunate to live in a time where AI is pushing the bounds of what is possible. It makes me question what I think is real, and why.

If I saw this a year ago, there is no way that I would have believed it was real. There have been too many "AI Generated" scripts/books/etc that turned out to be heavily edited by the author. This one feels a little too perfect, considering the author states this is largely what came out of the model on their first pass.

Today, I lean towards thinking this is real. Having used copilot and Midjourney for the last few months, I know that my "what is real" radar is completely off. Honestly, if you told me that many of the things I have generated via AI were actually a VERY fast fiverr worker, I might believe you.

What a time to be alive!

mavu2• 4 years ago

No, she didn't have a chat with herself, and not even with an AI.

I hate this complete and utter misrepresentation of facts.

GPT is not an AI. It does not talk. It certainly does not understand.

I wish AI researchers would spend some effort to educate the public instead of bathing in all the attention.

Guthur• 4 years ago

Here here.

But there is also a broader absence (destruction?) of our classical understanding of thought and other domains such as classical objective reality and the platonic domain of ideas.

Not yet sure if this is some weird post modernist view or intentional dummying down, maybe two sides of the same coin?

But at least from what I can observe there is little push to teach people how to think.

carabiner2• 4 years ago

*hear hear

grepLeigh4• 4 years ago

Even if the headline isn't accurate by HackerNews technical standards, don't under-estimate the power of believing this kind of experience.

For example, "What Happened to Make You Anxious?" by Jaime Castillo proposes a similar technique for addressing anxiety. The technique goes like this:

Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.

Sure, GPT-generated text isn't actually equivalent to talking to yourself - but you can see the utility for therapeutic applications like this.

narrator• 4 years ago

Think of a traumatic moment in your past, where you wish you could go back in time and comfort or offer yourself advice. Then imagine yourself stepping into the frame of your memory and giving your younger self the support you needed in that moment (but lacked). This might sound woo-woo, but the efficacy is documented.

This is how the Internal Family Systems psychology technique deals with past trauma and it has been around since the early 1980s. No AI required. To go that deep though, it helps to have a therapist trained in the technique to guide the process.

visarga3• 4 years ago

GPT is like a universal human template that can be prompted with a persona of your liking.

eagsalazar2• 4 years ago

I've heard this sentiment before (that AI != ML, always weirdly hostile) but I've also heard key figures in the AI world say strongly that ML of any kind is a subset of the broader AI umbrella. Are you sure it is strictly wrong to refer to this example as AI? Also why are you using such strong words? I'm genuinely curious why there is so much emotion when people, maybe, misuse these terms.

carabiner2• 4 years ago

He's going for a middlebrow dismissal. This is like the "bitcoin isn't money!" or "twitter is not a serious medium for communication!" oldschoolism that some HN traditionalists defend in spite of real world application. At the end it doesn't uncover a salient point but gets stuck on semantics to shut down an entire idea.

grepLeigh4• 4 years ago

I'm not the OP, but have been a data person long enough to hazard a guess.

If your job is an engineer or scientist, the term "AI" is basically a synonym for "unrealistic executive expectations." That's a super triggering and stressful situation to be in, especially early career.

I've actually had a CEO describe expected output of my ML team as "magic AI shit" - I bet you can imagine the team's reaction and tone. I'm reading the same strong emotions and frustration here.

The good news is you can always course-correct expectations with communication. I've come to love talking about AI with people who are only somewhat technical, because their wildest dreams are sometimes totally do-able with some duct tape and fine-tuning.

airgapstopgap• 4 years ago

Do you have some principled definition of what AI is or what it means to "understand"? GPT-3 outputs certainly show enough high-level non-scripted information processing to qualify as "understanding" in my book – not human understanding, clearly, but an artificial one.

AI researchers do in fact educate the public. [1] They face serious pushback from obscurantists like Hofstadter [2] and Marcus who refuse to commit to any definition and instead go by hunches about what architecture or data representation can support "understanding" and what certain behaviors indicate.

1. https://www.economist.com/by-invitation/2022/06/09/artificia...

2. https://www.economist.com/by-invitation/2022/06/09/artificia...

erehweb• 4 years ago

I dunno. ELIZA is not an AI, and doesn't understand anything, but colloquially it makes sense to say people chatted with ELIZA. https://en.wikipedia.org/wiki/ELIZA

belugacat• 4 years ago

It is her performing talking to herself. She's as much "talking to herself" than if she had written and then performed a play about talking to herself. The computer/AI aspect of it is "just" a prop for the performance.

leesec• 4 years ago

can't believe people are still so sure AI's can't understand something when they've clearly demonstrated many times now they understand a whole lot

bananamerica• 4 years ago

I know this is a technical forum, but you are allowed to appreciate the subjective meanings an artist ascribes to technological output, as well as the sentiments it provoked.

mastazi• 4 years ago

To me, it sounds like you are conflating AI and AGI https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

The way I and many others use the term "AI", is based on the understanding that AI is an umbrella term that includes modern deep learning as well as the broader field of machine learning, and also the other pre-machine-learning types of AI such as symbolic AI, expert systems etc.

If the stuff running on Lisp Machines was referred to as AI, I don't see why GPT-3 can't be labelled as such.

debugnik• 4 years ago

To me, "an AI" implies an autonomous agent, even if it is dumb. But many of those you listed are techniques for AI, not AIs themselves. The same goes for these ML models: most are techniques or components suitable for AI, not AIs themselves.

I'd consider these language models AI if they initiated conversations themselves or used conversation as part of a plan to achieve some objective. But so far, game bots are more deserving of the title than these trendy language models and painting generators, IMO.

halpmeh• 4 years ago

I think you're committing a yet-unnamed logical fallacy that I see a lot on HN. I'm going to call this "the logical fallacy of uncharitable interpretation." The basic premise is this:

Take an assertion like X implies Y. Unfortunately, due to the imprecision of language, X may have multiple definitions, e.g. D1 and D2, such that D1 implies Y but D2 does not. The fallacy of uncharitable interpretation is to take a definition for X such that the stated premise does not hold, even though there exists another definition where the premise would hold.

In this case, large ML algorithms are commonly called AI. Even the company that made GPT-3 is called OpenAI. Even you, yourself referred to them as "AI researchers." What do "AI researchers" produce if not AI? Finally, "talk" can colloquially mean to use words in an prompt-response pattern.

dvt3• 4 years ago

I'm not going to downvote you because I think you're respectful and thoughtful, but I want to vociferously disagree. These clickbait blogs know what they're doing. There's been so much contention about what is and isn't artificial intelligence (including the Google weirdo that thought it was sentient earlier this year), that just about anyone semi-educated on the subject knows how GPT and related models work. We know it's smoke and mirrors; we know it's plagiarizing and combining other, third-party, materials half the time.

She's not "having a conversation with her younger self" and this is incredibly disingenuous and misleading, per @mavu's point. More importantly, it doesn't really educate the uneducated or move the technology forward in any meaningful way. It's pure garbage clickbait nonsense. I regret giving them the impression.

planetsprite• 4 years ago

Ok, then according to you What is AI? What is "talking"? What is "understanding"?

msla• 4 years ago

Stuff a lot simpler than GPT-3 has been called AI by people much more accomplished than you for a lot longer than you've been alive. The term is not ambiguous, "Intelligence" does not exclusively mean human-level or human-like intelligence, and trying to redefine it now would only cause confusion.

knaik942• 4 years ago

I wonder how well social media or text message logs would work. I never journaled but I have decades worth of chat logs. I have conversations with people over with over 600k messages between us.

I am not familiar with the OpenAI workflow, but KoboldAI is a community open source implementation to interface with different AI language models locally or online. There are links to premade notebooks of open sourced models you can run different on Google Collab. [1] I think finetuning models offline is still too expensive, but there is some progress and is possible via collab.

KoboldAI has a chat mode setting where it properly generates and continues a conversation between two people. The models I ran were the smaller ones, but there's larger models available on huggingface like FB's OPT 30B. Even the smaller models I could fit into 8gb vram were coherent enough to be impressive.

I am surprised I haven't seen more of is a chat bot based on writes who have passed away. I imagine an Aristotle chat bot would be pretty interesting and avoid issues of copyright.

[1] https://github.com/KoboldAI/KoboldAI-Client

nick_m• 4 years ago

I'm curious about trying something like this myself - does anyone know which GPT-3 model she used? On their site, it looks like I have a choice of Ada, Babbage, Curie or Davinci. I'm new to GPT-3 - assuming that she started with a "base" model and then, trained it using her journals.

knaik942• 4 years ago

She added a small write up, it was Davinci 2 as 3 wasn't out. https://twitter.com/michellehuang42/status/15977029748891443...

ralfd• 4 years ago

Thanks for the link. That sounds more like prompt engineering? If I understand that correctly it is providing short journal entries (1K words) and GPT3 is imitating the vibe of that (or whatever it does). But it is not “training” a Modell on all the journal text.

visarga3• 4 years ago

I am also interested in this. For example how should we best formulate the input? Just our own messages, or including the parent message, or the whole chain to the top and the linked article. I think in the future we will have easier ways to train a persona.

ccleve• 4 years ago

This is a conversation a man had with his younger self. It's way, way better: https://m.youtube.com/watch?v=XFGAQrEUaeU

Aeolun2• 4 years ago

I'm not sure. The responses the AI gives seem to me like they could have been copied from any conversation about childhood with anyone (which isn't surprising, since the base is GPT3).

tartoran3• 4 years ago

Why would it after all? It’s all mimicry without understanding of any context. It’s going to become more and more real and more convincing but beyond that there is still nothing intelligent, just fascinating stochastic immitation.

Aeolun2• 4 years ago

I’m not entirely certain humans are any different though. We’re essentially very sophisticated pattern matching machines, based on our past experience.

megablast• 4 years ago

The text she writes are vague too.

bsenftner• 4 years ago

This is Art because of the cunning use of ambiguity: by training on a subset she's framed the potential responses, and presenting as a conversation with her younger self the stage is set, all that is necessary for the idea to take flight is there. And it takes flight. This is how Art and whatever we're calling AI meet commercialism, and sell clicks. What she evokes is going to generate a little industry; just watch it unfold.

itronitron• 4 years ago

I foresee a hip young startup 'diaryly.com' in the near future to help people improve their conversations with themselves.

Imnimo2• 4 years ago

I really dislike the trend of using the phrases like "I trained an AI chatbot" to mean "I prompted an off-the-shelf language model".

jameshart• 4 years ago

What is the minimal level of effort required for it to count as 'training an AI chatbot' for you?

Isn't 'one' shot' or 'few shot' prompting a kind of training?

Isn't the result, purely from the point of view of the interacting user, functionally a chatbot?

Imnimo2• 4 years ago

I would say that training implies an update of the weights, at a minimum. I would accept fine-tuning GPT-3 as training, but not prompting.

I think that "chat bot" is less objectionable, but it presents a bit of a misleading picture to a casual reader. I think people ascribe a lot more agency to a "chat bot" than they would to a "next word predictor".

josefx• 4 years ago

What is the minimal level of effort required for it to count as 'training an AI chatbot' for you?

Isn't GPT-3 already capable of acting like a chatbot out of the box? The article implies that it was trained to be like a younger Michelle from the journal entries, but all the responses presented seem to be utterly generic. Talk about dreams and hardships but not anything concrete, no mention of friends and family, locations or activities. I don't know enough about GPT-3 to say if the training affected word choice, but it doesn't seem to have any discernible depth at all.

stolenmerch• 4 years ago

I agree, but at least it isn't "I forced a robot to read my diary", which is something I've seen.

rco8786• 4 years ago

fed in about 13,000 characters before reaching the maximum threshold

Wait, certainly that’s not even remotely close enough to enough data to train a “past self” AI on. Even if we assume that by “characters” she meant words that’s just not very much text.

throwntoday• 4 years ago

Yeah my bullshit meter is going off. Cute fiction though.

krageon• 4 years ago

You assume that your personality cannot be (semi-)successfully simulated as something that only barely diversifies from everyone else.

staticman2• 4 years ago

If you creatively define "semi‐successfully" then any simulation can simulate anything.

nomagicbullet• 4 years ago

Just yesterday we had a post about how Americans are spending more and more time alone[1]:

in 2021, the average American spent only two hours and 45 minutes a week with close friends (a 58 percent decline relative to 2010-2013).

I understand some people might feel uncomfortable about "AI chatbots" (about their inaccuracy, or "fakeness"). However I can't help to think that the future of humankind will be plagued by artificial entities that we will interact with as we do with other humans. It might be in the form of simple ELIZA models, some ML artifact or full-on conscious intelligences.

I suspect there will be huge benefits to virtual companions to help with this crisis of loneliness that afflict us. In any case I hope humans will evaluate these technologies by the value they derive from them.

[1] https://news.ycombinator.com/item?id=33755243

Gigachad• 4 years ago

I think the uncomfortableness comes from the fact that just about everyone acknowledges this scenario is inevitable. Of corse we will all be replacing friends with chatbots because that will be easier and eventually they will work even better than real friends.

But what kind of world are we going to end up with when this happens. What is the result of giving everyone chatbot friends who are all perfectly agreeable. And who probably slip in product placement/ads.

aaomidi• 4 years ago

I really hope virtual companions aren’t centralized or if they are there is extremely strict regulations around the data.

Imagine the company changing policy and you being locked out of your virtual companion? That should never happen.

Tao33002• 4 years ago

Just read the Young Michelle lines and you can see that they are very ELIZA-ish. Most of them could have been mildly sympathic-sounding responses to any number of inputs.

kqr• 4 years ago

The one impressive bit that ELIZA would not do is understand some of the semantics of the prompts. Like you can ask it to ask a question, or write a letter, and it would get the format of the response right.

But yes, I agree. I'm not sure how much the journal entries really contributed other than specific turns of phrase and general topics of discussion.

donatj2• 4 years ago

I've got all my AOL from around 2000-2008... This could be interesting.

azinman2• 4 years ago

How? What / where was that stored?

dustincoates• 4 years ago

I can't speak to the above, but I know Trillian would save them as plain text on device.

donatj2• 4 years ago

AIM stored chat as HTML log files. When I moved to Adium I kept all of those as well.

Combine that with being an extreme data hoarder and having full backups of basically all my computers 2000 -> now on my NAS and I've got almost everything…

barbariangrunge• 4 years ago

Reminds me of those people who marry their dolls.

GPT-3 is not a person. The biggest risk with new ai tech is that it will learn to win us over, seeming like people, when there is nothing happening on the inside but number crunching. Earning empathy despite having no feelings.

an_ko• 4 years ago

I'm pretty sure if I look closely enough, I'm nothing but numbers myself. My feelings don't exist, they're just side effects of evolved processes that made my ancestors survive, generated by a neural network too complex to debug. Compared to GPT-3 though, I'm better at pretending to be a person.

(The above was generated by GPT-3.)

(No it wasn't.)

Lapsa• 4 years ago

I've yet to see a good chat bot. one way or another - they all remind me of Microsoft's Tay.ai (!) at best

xerxesaa• 4 years ago

This sort of reminds me of the Black Mirror episode "Be Right Back"

pgt2• 4 years ago

In the days when Sussman was a novice Minsky once came to him as he sat hacking at the PDP-6.

"What are you doing?", asked Minsky.

"I am training a randomly wired neural net to play Tic-Tac-Toe."

"Why is the net wired randomly?" inquired Minsky.

"I do not want it to have any preconceptions of how to play."

At this, Minsky shut his eyes. Sussman asked his teacher, "Why do you close your eyes?"

"So that the room will be empty."

At that momment, Sussman was enlightened.

isoprophlex2• 4 years ago

I never got that one... care to explain the wisdom it is supposed to convey?

bryanrasmussen2• 4 years ago

well I would suppose it meant that if you close your eyes so that the room is empty you have a misconception as to the state of reality, there is not actually a connection to the room being empty and your eyes being closed it just seems like there is to you.

thus there is not actually a connection between the neural net being randomly wired and it not having any preconceptions as to how to play, it just seems like that to Sussman.

I have never found these zen koan things very interesting though. Also I think there is much more likely a connection between a neural net being randomly wired and not having preconceptions than closing of eyes emptying out rooms.

Slix• 4 years ago

I thought he was rudely telling the novice to leave.

EarlKing• 4 years ago

Wiring a neural network randomly does not eliminate preconceptions, but rather you now have a random set of preconceptions... which is precisely equivalent to closing your eyes in hopes that the room will be empty (which obviously it won't be).

pgt2• 4 years ago

The artist tweeted [^1]: "this way, i could accurately simulate what it would be like to talk to my childhood self, based on real data sources during that time period vs trying to imagine how my younger self was / how she would respond, and risk bias from projections from my current self"

The Minsky story is a comment on the ridiculousness of the idea that you can train a model on your journal to "avoid bias" and "accurately simulate" what you were like as a child.

[^1]: https://twitter.com/michellehuang42/status/15970055048693719...

swyx• 4 years ago

can someone give some technical pointers on how to do this? her thread is very light on the details. is she using https://beta.openai.com/docs/guides/fine-tuning ? what are the best practices and limitations for doing this?

up2isomorphism• 4 years ago

Seriously we are spending tens of billion on this kind of crap and feel good about it? The only use case I can see for this is scam and pig butchering schemes.

adamredwoods• 4 years ago

When HARLIE Was One

twayt• 4 years ago

How do you know this is actually from GPT-3?

skc• 4 years ago

Ahh, a fellow cynic.

I was wondering exactly how she entered her presumably hand written notes from childhood into the GPT-3 playground.

Did she just retype all of them? Did she hand pick certain passages?

I ask because I have a feeling GPT-3 would return roughly similar results without the journals.

nigerianbrince• 4 years ago

Coming up next, I trained a sexbot ai using my vlogs to have sex with myself.

Artist feeds childhood diary into GPT-3 to have a chat with herself

Related Stories