How to Train an AI Image Model on Yourself

https://www.coryzue.com/writing/make-ai-pictures-of-yourself/

by aberoham • 1 year ago

207 42 1 year ago

ge966• 1 year ago

What I want is to be able to feed in a bunch of videos and generate an animatable (from talking) 3D face from that data. I suppose you in theory only need 3 images (front and sides). But mapping pixels to motion is interesting (facial expressions).

There wouldn't be depth data so it would be inferred from shadows

ilaksh• 1 year ago

Replicate has Hunyuan video training now. https://replicate.com/blog/fine-tune-video

Also, Kling 1.6 Elements works pretty okay if you use the same person/face for each element.

Kling also has lip sync.

Or this lip sync with replicate: https://replicate.com/bytedance/latentsync

Or there is HeyGen or D-ID or Synthesia, or tavus.io for full interactive digital twins.

ge966• 1 year ago

Thanks

timdiggerm• 1 year ago

Why do you want to do that?

ge966• 1 year ago

My case is not directly nefarious, for example an old popular YouTuber that streamed in the early 2000s taking their content and making a model of them for personal use like a 3D chat bot but with that person's quirks

Edit: when I say "nefarious" I mean you can use that tech to impersonate someone (eg. political reason) but for my case it's more the creeper type cloning someone for personal use eg. Replika

Tangent, the holo vtubers industry is interesting since they build up these characters with some unique persona/theme and then people follow that specific model, they could make themselves into an AI easily since it's a rigged 3D asset but of course it would be boring compared to the real thing

manishsharan• 1 year ago

This is fantastic but now you need to train a model to detect AI generated images from actual photos. Then of course , a model to beat the detector model and then a model to catch the model that beats the detector model and so on.

Thank you from people holding NVDA.

beng-nl• 1 year ago

You may have re-invented GANs :-)

petercooper• 1 year ago

Replicate does make this particularly easy while still being somewhat developer focused. I've used it for a few people in our group chat so we can make silly in-joke memes and stuff and the results are quite stunning. Replicate then offers the model up over a simple API (shown in the post) if you wanted to let people generate right from the chat, etc. Replicate is worth poking around a bit more broadly, too, they have some interesting models on there (though the pricing tends not to be very competitive if you were going to do it at scale.)

deadbabe• 1 year ago

I’m imagining something where an influencer trains AI to make and post images of themselves on social media, then the influencer dies but the AI keeps going forever.

ge966• 1 year ago

The impact is kind of interesting, how do you know someone's legit, the person doing basejumping or whatever

Thanos/NFTs: where did that take you? right back to me

Thinking hardware with built in chain interface for proof

Oh man dating apps too

That's true love though, two people meet up IRL they're both like wtf who are you

isoprophlex• 1 year ago

I did this for our beloved, dead cat... On replicate, too. I loved the results, until at one point I suddenly got really creeped out about the thing I was doing.

ryandvm• 1 year ago

This is going to be big business I think. I have probably sent hundreds of thousands of emails, texts, chats, etc. It would be well within the realm of possibility to train an LLM on a loved ones communications corpus and allow you to chat with "them" after they're gone.

Possible? Yes. Convincing results? Probably. Good idea? I doubt it.

mipmap04• 1 year ago

Oh man, I did this with my dad's voice after he died and set up a thing where I could talk with an LLM-backed assistant and have it respond in his voice and mannerisms. It was a very weird coping and grief period and I ultimately hit a point where I got really weirded out about what I was doing.

slig• 1 year ago

I remember seeing it here on HN that someone did that with a group chat and it would reply as each friend.

knicholes• 1 year ago

This is exactly what I'd want to do for my "smart urn."

portaouflop• 1 year ago

I think that was 1:1 a black mirror episode

waspleg• 1 year ago

Literally a Black Mirror episode.

oskarkk• 1 year ago

This reminds me of paintings in Harry Potter.

numpad0• 1 year ago

Is it going to be good for your sanity? I very much doubt it.

ramoz• 1 year ago

FYI: For Flux, there is a lot more power in the text-encoder & you can prompt with more meaningful and comprehensive sentences. Thus, less of the traditional comma separated & concise phrasing we saw in stable diffusion.

You should do the same with your training images. Caption everything you do not want the model to remember as "you" (what you're doing, wearing, accompanied by, accessories, etc).

AuryGlenz2• 1 year ago

You can do that, but you can also just do your name (or, I do two prompts - one the name, the other “A photo of _name_” as that makes it easier to prompt non-photographic images). This has the bonus of not training that particular shirt or whatever in, so if you prompt for something similar it won’t be identical.

That said I usually train with about 100 really varied images of the person so it tends not to overlearn any other particular thing.

DoodahMan• 1 year ago

is something like this possible to do with video yet?

dvrp• 1 year ago

you can use some providers like kling or platforms like krea.ai to do consistent characters with one frame

AuryGlenz2• 1 year ago

Yeah, with Hunyuan video.

thefourthchime• 1 year ago

I did this a while back, though it was pictures of my wife in lingerie.

- I asked grok to generate a list of racey prompts. - Has replicate generate them via script. About 10-20% are very poor, I filtered those out manually. - It also has NSFW guardrails, but a simple retry or word juggle gives you a chance to get around it.

I think I spent $10

Der_Einzige• 1 year ago

There is a parallel "underground" AI research world of stuff like this, with it's hub on "civit.ai" instead of huggingface.

Often the innovations from that world are ahead of mainstream AI research by years. You should see what coomers did for LLM sampling in order to get over issues with "slop" responses just for their own pervy interests. This is a full several years before the mainstream crowd ever cared.

ok_dad• 1 year ago

Porn has always pushed the boundaries of media on the internet. I don't know why people are surprised! Since sex is something nearly everyone does, it would make sense that a lot of human progress were the result of trying to integrate sex and whatever new tech is out there at the time. I am sure a hundred years ago some inventors were pushing the boundaries of motors in sex toys, and in another hundred years some other inventor will be pushing the boundaries on putting sex in holograms.

DrSiemer• 1 year ago

It's kind of annoying that some of the best models out there have a tendency to produce very not safe for work results.

Look mom, I can make some cool astrology images for you! Whoops, that's boobs. That too. And this one. Ehh, hold up, I need to add a pile of negative prompts first...

njx• 1 year ago

Thank you for sharing. Is there any model that can help train convert pictures into cartoon or flat vector illustration?

Our_Benefactors• 1 year ago

Look into using img2img in stable diffusion

m463• 1 year ago

I had set up automatic1111 a while back, and I believe the webui let you your image generation have a starting image. It's kind of fun to have a cartoon of yourself based on an image.

How to Train an AI Image Model on Yourself

Related Stories