Welcome to Waifu Labs v2: How Do AIs Create?

https://waifulabs.com/blog/ai-creativity

by Cixelyn • 5 years ago

455 213 5 years ago

liuru17• 5 years ago

Hey HN, one of the team members here!

I hope you all enjoy playing with the new and improved generator! We've been hard at work improving the model quality since the last time the site was posted[1]

As both a professional fantasy illustrator & software engineer, I find the concept of AI creativity so fascinating. On one hand, I know that mathematically AI only can hallucinate images that fit within the distribution of things that it's seen. But from the artist perspective, the model's ability to blend two existing styles into something so distinctly new is so incredible (and not to mention also commercially useful!)

Anyways, happy to answer any question, thoughts, or concerns!

---

[1] https://news.ycombinator.com/item?id=20511459]

hypertele-Xii• 5 years ago

Why do stuff like this never come down from the web? I'd pay for a program I could download and use with my own image files.

Afforess• 5 years ago

Same reason the Coca-Cola recipe is not published nor made freely available by the Coca-Cola corporation.

Gigachad8• 5 years ago

They tend to require specific hardware like a NVIDIA GPU. As well as having an ever evolving large model file which they will want to frequently update. Some tools certainly have had offline versions but I guess not many people are interested in setting it all up and are happy with an instant web ui

zozbot234• 5 years ago

You just need to code up your own model architecture and then train it on your data using some established ML framework. The first step is where well-chosen priors can make a real difference wrt. your end results.

liuru17• 5 years ago

While our model is not public, there are good resources online for playing with your own images!

Like this one by fast.ai!

https://docs.fast.ai/vision.gan.html

dimgl2• 5 years ago

You mention it took two weeks to get to the point that we see in the article.

Does this mean two weeks of development, or two weeks to generate the images we're seeing? Or maybe did you train the model for two weeks? That point just wasn't exactly clear for me.

liuru17• 5 years ago

2 weeks to train the model!

Development took on-and-off roughly 2 years to achieve the quality you see today.

darkengine• 5 years ago

Is the code or any of the models available to the public? I'd love to mess with this on a local GPU cluster.

liuru17• 5 years ago

Not at the moment! A similar project that I really admire is public, though!

https://www.thiswaifudoesnotexist.net/

2bitencryption• 5 years ago

Firstly, amazing work.

My question is, how do you figure out how to parameterize "Same character, different pose" / "Same character, different eyes" / "Same character, different gender" / etc?

My (super limited) understanding of GANs is that they slowly discover these features over time simply from observation in the data set, and not from any labels.

So how could you make e.x. a slider for head position, style, pose, etc? How do you look at the resulting model and figure out "these are the inputs we have to fiddle with to make it use a certain pose"?

You mention it a bit in this section, but I didn't fully understand: "By isolating the vectors that control certain features, we can create results like different pose, same character"

And I assume the same step needs to be done every time the model is retrained or fine-tuned, because possibly the vectors have shifted within the model since they are not fixed by design?

liuru17• 5 years ago

Yes, your understanding is correct!

You can think of it like coordinates on a many-dimensional vector grid.

We craft the functions the functions that will illuminate sets of those points based on a combination of observation, what we know about our model architecture, and how our data is arranged.

And yes, when the model is retrained, we have to discover them again!

flor1s• 5 years ago

Not affiliated with this project, but there is a gazillion different variations of GANs. Most just change the adversarial loss to improve the learning rate / quality, but others focus on architectural changes, such as StarGAN, Pix2pix (conditional GAN), CycleGAN, MUNIT, etc. It's really a fascinating field.

rackjack2• 5 years ago

I LOVE that "horror". Reminds me of some of the art I've seen on album/single covers. Any chance of letting people access that kind of intermediate step? (Though I know it's a niche as hell use case).

liuru17• 5 years ago

Ah yes, the fine line between charming anime character and lovecraftian horror

There was such popular demand for these "horror" images that we made them part of the generation in V2! If you refresh enough on the webpage, you can find some horrors!

kregasaurusrex2• 5 years ago

I purchased a waifu from your vending machine (loved the blog post!) at Gen Con in 2019, but can't see the saved model in my account. Is there a way for me to get a v2 generation?

liuru17• 5 years ago

Welcome back!

We're currently working on the data migration from V1! As long as you are using the same email as you did in 2019, you'll be able to see the image again!

As for a V2 generation, sorry, because the models are different, you'll have to discover a similar image again, if you want a V2 version!

kouteiheika• 5 years ago

What are the terms of use for the images generated through your website? I'm guessing any commercial use is forbidden? It would be nice if you could formally spell it out on the website.

JetAlone2• 5 years ago

I don't think there's any powerful enough way to stop people from generating one and then tracing over it to create their own linework, and customize things like the colouring and shading. The more broadly AI is able to create, the more niche and obfuscated directions human co-creators could take its products in.

searchableguy• 5 years ago

Do you plan to provide an API to generate waifu?

I think I could use this for a project.

liuru17• 5 years ago

In the future, perhaps! This is a popular request, so we are thinking about ways we can do this.

unobatbayar2• 5 years ago

The quality and style is mindblowing! What data did you train?

Bombthecat2• 5 years ago

The more interesting question: What about the sources for the AI to train? How are those artists paid? Do we need to pay them? Or if it used by an AI as train data, we just say: Its like a human learning?

lynzrand• 5 years ago

As a rough guess, I think it might be trained on the Danbooru archive dataset, since it's the largest anime picture dataset we can get today.

https://www.gwern.net/Danbooru2020

liuru17• 5 years ago

The first iteration of our model was built off of this amazing public dataset:

https://www.gwern.net/Danbooru2020

Though now we have made our own :)

wodenokoto2• 5 years ago

Naïvely I thought Waifu generator was just “some guy having a laugh” fine-tuning a model off of hugging face, but reading through the comments here, it is obviously a much, much bigger enterprise.

Can you talk a little about team size, work process, funding and revenue stream? I think the effort required for such an undertaking is vastly underestimated by readers.

Cixelyn6• 5 years ago

Right now it's a small team of 6 people, and we have a bit of funding + compute credits to train models. There's a bit of revenue from some past projects and AI-consulting, but we're mostly betting big on our new AI-powerd mobile title Arrowmancer[1].

I think the effort required for such an undertaking is vastly underestimated by readers.

Haha for sure. Hosting a real-time ML model for people to do sub 1-second inferences at HN-load scale is definitely nontrivial.

[1] https://arrowmancer.com

hansel_der2• 5 years ago

Naïvely I thought Waifu generator was just “some guy having a laugh”

same here. what's naive about it?

not to badmouth the undertaking, but wtf is this doing on HN?

ridaj• 5 years ago

Fascinating... Thanks for sharing

A couple questions:

1) I didn't really understand how you went about identifying what vectors of the latent space stand for various things, like pose or color. Did you train one of the AIs to that effect, or did you manually inspect a bunch of vectors, twiddling through them one by one, did to the outcome?

2) If one were to train an AI to the same level using commodity cloud services, what's the order of magnitude cost that you would pay for the training? More like $100, $1,000, $10,000 or $100,000?

liuru17• 5 years ago

1) It was mostly manual, though AIs were useful in certain filtering tasks.

2) Depends on the quality you are seeking. If you only want one run of a similar, off-the-shelf model, around the 1000s is enough. But at the number of iterations you have to run to build your own and improve results, you probably need about 100k.

To tackle this problem, we built our own supercomputer off of parts we bought off of ebay, though I can't say I recommend that route, because it now lives in our living room.

thyrox• 5 years ago

Roughly speaking how much money did you invest into making this? Just curious if this is something an indie hacker can hope to do one day OR do you need some deep pockets to make a site like this?

Cthulhu_3• 5 years ago

With the game you're building, are the character portraits generated once and that's it, or do you plan on making them dynamic or frequently updated?

I've seen a number of mobile games that just get flooded with characters; this tool looks like it could be used to automate that process. It could be combined with AI-generated character profiles as well, creating an 'infinite' character roster in video games.

GoblinSlayer5• 5 years ago

Would you try to create a new style? Train the discriminator on the score tag of danbooru dataset, then use it to rate the generator's style, this way it should be able to create a new style.

Terry_Roll• 5 years ago

I wonder what an AI trained to spot deepfake Waifu's will detect.

In humans, things like the pupil can be the give away.

https://www.newscientist.com/article/2289815-ai-can-detect-a...

yccs27• 5 years ago

This is a super interesting question, given that the generator model is trained to fool the discriminator, which is also an AI.

oneoff786• 5 years ago

Highlight the pixels with high sharp values. Should be doable.

CuriousCosmic• 5 years ago

Something I was wondering but couldn't find on the site: What is the license for the generated works through the project?

YeGoblynQueenne4• 5 years ago

Hello and thank you for answering questions. The following is a quote from your article:

> It is interesting to note that from this process, the AI is not merely learning to copy the works it has seen, but forming high-level (shapes) and low-level (texture) features for constructing original pictures in its own mental representation.

Can you explain what you mean by "mental" representation? Does your system have a mind?

Also, why are you calling it "an AI"? Is it because you think it is an artificial intelligence, say like the robots in science fiction movies? Is it capable of anything else than generating images?

xg15• 5 years ago

Not OP, but I wonder if the process would be in some way comparable to rigging a 3D model. There is well, you usually have some high-level input parameters, which influence joints on a predefined skeleton, which in turn determines the position of individual vertices in the 3D body. Finally, the 3D shape is used to render the actual pixels.

On each step, high-level parameters are combined with predefined weights to produce a more low-level output.

Seems, a similar transformation is going on here, except that the weights and the structure are somehow learned on its own.

simonebrunozzi• 5 years ago

So neat! Where are you based? Boston, I assume?

Is there an email to reach out to you or someone in the team? ($HNusername @ gmail)

Cixelyn6• 5 years ago

San Francisco! Just sent over a ping!

tedmcory77• 5 years ago

Who would someone speak with about licensing things made using waifu? My email contact is in my profile...

kadokaelan2• 5 years ago

The explainer video about gans is top-notch! Excited for Arrowmancer!

Cixelyn6• 5 years ago

(link for the lazy: https://youtu.be/Pab8pG5WbXQ)

Thanks so much! It's done by our fantastic animator[1]!

GANs are quite interesting and we didn't see many approachable explainer videos targeted at lay people, so we decided to make one ourselves!

[1] https://twitter.com/bumblingbeebo

Bombthecat2• 5 years ago

Is there a gameplay video? Or at least screenshots of arrowverse?

slimsag• 5 years ago

For the author: there's a small typo "Discrimniator" instead of "Discriminator" in the video at 1:11

One thing I was confused by: the video says the discriminator "AI" is trained to detect true vs. generated results, with the hope the generator becomes good enough to fool the discriminator. But why is the discriminator useful, then? Couldn't you just tell generator "AI" whether the result it produced was true or not?

I think the answer is.. you don't want just a perfect recreation of the training data you gave to the generator, instead you want the generator to produce variations of that training data, so there's a "how would you know if it's 'a true result' / good enough?" problem. So the discriminator is useful because it's not a direct comparison, but rather a "this looks approximately good enough" comparison of the true vs. generated result.

This all makes me wonder: what sort of data set needs to be fed to the discriminator to train it? Is it some sort of "true image" and "true image w/bad alterations (e.g. lines, scratches, etc.) to it" data set?

liuru17• 5 years ago

Thank you for the typo!

Indeed, it contributes to the variations problem.

also: If the discriminator starts off perfect, then the generator can't learn to be better.

Sort of like a human learning to play chess: If you start off with top-tier opponents that crush you, then you don't have a gradient to learn from. Instead, you need players at your own level to grow your skills.

FeepingCreature4• 5 years ago

If you think about how humans draw things, there's a repeated process of "create content" to "inspect for issues" back to "create corrections", "inspect for issues" etc. It seems that generation and criticism are two rather distinct skills, which is why GANs, generative-adversarial networks that contain both a "generator" and a "critic" network, end up making fewer mistakes and learning distant correlations better.

drummojg• 5 years ago

The game is an interesting mix of mobile fun with the concept for El-Fish. https://www.wired.com/1993/02/maxis/

ve55• 5 years ago

Links to related projects in anime art generation for those interested:

Waifu Labs v2, referenced in this post (generate amazing custom anime face images): https://waifulabs.com (write-up is the above link: https://waifulabs.com/blog/ai-creativity)

This Anime Does Not Exist (AI-generated anime-style artwork): https://thisanimedoesnotexist.ai (write-up https://www.gwern.net/Faces#extended-stylegan2-danbooru2019-... and https://nearcyan.com/this-anime-does-not-exist)

This Waifu Does Not Exist (AI-generated anime-style faces): https://thiswaifudoesnotexist.net (write-up: https://www.gwern.net/Faces#twdne)

There's also a lot of literate on e.g. automatic manga coloration, auto-translation, image superresolution, anime frame interpolation, and much more. Worth checking out some places like https://old.reddit.com/r/AnimeResearch/ if you're interested!

the8472• 5 years ago

Note that TADNE, being based on newer models, generates full images rather than only faces.

numpad08• 5 years ago

This one too https://twitter.com/t_takasaka/status/1479784432697749513

spywaregorilla2• 5 years ago

Is there any way to approximate embeddings for a novel image?

novel meaning user provided, not generated by the model or in the training set.

liuru17• 5 years ago

Not at the moment in our tool, though this is an area of great curiosity and research for us!

spywaregorilla2• 5 years ago

Does the discriminator model translate the images into an embedding space of it's own? Could such a space be used to generate images themselves?

mensetmanusman• 5 years ago

I can’t wait for this technology to come to video.

Imagine a future where people can compile written scripts into Hollywood quality movies.

thewarrior• 5 years ago

It's already begun https://twitter.com/somnai_dreams/status/1477411531037937664...

FeepingCreature4• 5 years ago

I wonder what will happen when somebody combines a GAN with a feature recognizing network like the Tesla cars use, so it can use its own extrapolated map of the surroundings to stabilize its output as the camera moves around.

JetAlone2• 5 years ago

I think it will have its limits, but the possibilities for editing together, supplementing and modifying products from smaller AI modules should stretch out what you can do on a small budget.

ps91• 5 years ago

Amazing now I can create my own anime girl!

KaoruAoiShiho2• 5 years ago

I don't want to go all SJW on you guys, amazing work, but can you try to make sure there's an inclusive array of starting faces please? Talking about things like skin tones, thanks!

liuru17• 5 years ago

Indeed, we spent 2 years working on this!!!

It's an extremely hard research problem, because darker skin tones account for only about 0.3% of all anime art produced in the world.

We have employed an absolutely exhaustive array of art and data science tricks to give the model the ability to draw darker skin tones, though they are underrepresented. The results that you see today are the culmination of many months of careful tuning!

It's not definitely perfect, but from a data science perspective, this situation can't be rectified until the art world makes a shift.

Personally, I hope that more art representing dark skin tones will be created in the world!

KaoruAoiShiho2• 5 years ago

Thanks, happy to hear that you guys are on top of it.

pfisherman2• 5 years ago

I can’t understand why the GP comment is flagged. If you can look past all the “culture war” stuff, this is pointing out some of the limits of algorithmic creativity.

It does not do well generating instances with features that are not well represented in the training dataset.

Compare this to human creativity. I suspect that fulfilling GPs request would be almost trivial for a human professional artist.

To be clear this is an amazing achievement, a creative use of the technology, and a positive contribution to the world. Pointing out limitations (i.e. areas with potential for future innovation) does not diminish it.

numpad08• 5 years ago

I don't quite understand the need to mandate racial caricaturizations in every modes of communication. I would find it upsetting if every pieces of texts and quotes were prefaced with an "origin" indicator to aid forming prejudices. Why have it in manga?

greenn• 5 years ago

Honestly asking, isn't this stuff usually Japanese characters? I dunno if theres even "data" they could use for other skin tones

userbinator• 5 years ago

On the other hand, you get a full spectrum of eye and hair colour.

numlock863• 5 years ago

Well, there is (or rather was) "ganguro" at least, which appeared in mangas an animes occasionally, too. Not sure how many people would be getting offended by that these days, though.

numpad08• 5 years ago

Anything irrelevant to plot or story building, such as nationality, are usually left unspecified. Readers often project their own identity into characters, and some people seem to find it odd that they do not encounter traits that differ from their own.

Ethnicity is sometimes incorporated, that is, some distinctions would be necessary if there was a documentary manga about a match in an Olympic Games played by teams from multiple parts of the world, and in that case an American players might be given smaller eyes or extra wrinkles in face, or African players might be colored darker than other characters, Chinese players could be drawn with slightly different shapes of chins, etc.

But the default is unspecified or an averaged, most simplified shapes and forms that the author uses in their own cognition.

newobj• 5 years ago

Anime has a pretty severe "representation of blackness" problem (https://www.youtube.com/watch?v=hi2_S6kBgIg is one video discussing this). I'm afraid to imagine what a model trained on that source content would generate.

Gigachad8• 5 years ago

This is extremely impressive. It’s the first GAN I have seen which lets you tweak the result in a meaningful way rather than being just random.

I think the speed that GANs have come in to the world has really shaken people up and it’s hard to process what this all means and what it will result in. Especially the ones which generate based on real people.

But the feeling this gives me, is what happens for the future of art. Sure, this example is no where even close to replacing real artists, but it’s already generating images better than I can draw after a year of practice. It does give me a feeling of “what is the point”. Which might be an irrational feeling, but I’m sure others feel the same.

zacmps• 5 years ago

We have been here before, and we certainly will again.

It was not so long ago computers bet humans at chess, yet people still play.

Gigachad8• 5 years ago

Yeah I totally get this is the same thing that has happened so many times before. But at least for chess, it was never a practical thing and was a competition. Like how the existence of aimbots does not ruin a game as long as you are playing against other humans.

For art it feels a bit different since it’s not competitive and more a practicality thing. Perhaps art will shift from placing individual strokes on an image and move to making creative directions for AI to resolve in to an image or enable more people to create labor intensive works like animation.

_trampeltier• 5 years ago

Or with the internet , Linux and open source. Suddenly you could downloads 1000s of apps, most much better, what I ever could do byself.

DavidPiper• 5 years ago

I've been thinking about this a bit lately, incomplete thoughts ahead...

Yes, people still play, but they no longer create.

With the exception of Adversarial attacks on particular algorithms, no human is creating new Chess theory, discovering new openings, for example.

As a game, challenge, competition, social activity, chess is alive and well.

As a creative endeavour, or vehicle for discovery, Chess is solved. It is no longer an art of its own.

We're part way through this transition now with Go as well. New opening theory, new joseki, new strategies are being played by robots, and at the highest professional levels we are playing catch-up to understand.

jychang• 5 years ago

Well, digital art already allows you to copy/paste a layer and stuff that traditional art never allowed for. Imagine copy pasting a statue.

This is sort of the same thing on steroids. You can copy/remix previous art by feeding them into a ML model in training mode, and it will be massively utilized the same way ctrl-c ctrl-v is used, but it's a part of the toolset of art creation, not replacing it.

numpad08• 5 years ago

You can copy/remix previous art by feeding them into a ML model in training mode, and it will be massively utilized the same way ctrl-c ctrl-v is used,

Then you need attributions for said previous arts, all of it, at least going by texts of laws.

liuru17• 5 years ago

Haha, this is something that constantly ponder, while I am developing this product.

Though, the conclusion I've come to is that that hand-drawn art will always meaningful for humans, because it is born of the human experience.

An interesting example is the invention of photography, which at its time, was very good at doing the thing artists were doing back then (capturing likenesses)

But photography didn't replace art: instead, artists now use photographs to be more expressive, convincing, and make better art. In tandem, the widespread adoption of photography meant that more average folks could get their likenesses taken!

Personally, my skills as an artist has improved by quite a bit, after launching this product, purely because observing it offers some fascinating insights into how anime is created!

I hope that as an industry, we'll find better ways to create, and what we know to be the "best" art today will be even better in the future!

eezurr2• 5 years ago

One thing to consider is the number of artists employed by businesses. I dont have actual numbers but i know video game companies employs many. Businesses have different values than an individual human, and won't care if the art is made by a computer if its cheaper and passable. Theres also huge market for NSFW art. Whos going to know who or what made the image?

Comparing photography to hand drawn art is silly. They are two different mediums.

Your company could be the first to capture the market. I guess if you can sleep with the consequences of your work,who cares? Im not judging because if its not you, it will be someone else.

Personally i think we as a society need to step back and press pause and really consider the consequences of this technology, and even existing technologies.

If you become rich, could you set up a charity for all the future starving artists, if that future comes to pass? I dont want to live in a world where theres no room for human creativity.

Not an artist, just a concerned human.

bmitc6• 5 years ago

Art isn't about statical copying or drawing well. It's about human expression.

Gigachad8• 5 years ago

These generated pieces look extremely similar to real human art so they look very expressive to me. And you can even add your own input by hitting refresh until you get something similar to what you wanted.

Cthulhu_3• 5 years ago

I hear what you're saying, but at the same time, there's a ton of mass-produced, low-creativity art out there, where the objective is not to express oneself but to churn out products with the intent to make money. This is not just in artwork, but in other design based things like e.g. website designs as well.

adrianN• 5 years ago

It doesn't seem far fetched that in the near future an "artist" doesn't need to be able to draw at all. If GANs or the like become sufficiently tunable, I can express myself by telling the GAN what I would like to have drawn.

plutonorm4• 5 years ago

hahaha. Now everyone will get a piece of my existential ennui. Welcome to the club!

Filligree2• 5 years ago

But the feeling this gives me, is what happens for the future of art. Sure, this example is no where even close to replacing real artists, but it’s already generating images better than I can draw after a year of practice. It does give me a feeling of “what is the point”. Which might be an irrational feeling, but I’m sure others feel the same.

There's a similar situation ongoing with fiction writing, by way of NovelAI. (And some competitors, but NovelAI is head and shoulders ahead of the pack. Thankfully; they seem to be the nicest of the lot.)

I'm a fairly prolific (fan-)fiction writer, and also AI enthusiast, so of course I jumped on that bandwagon as soon as I could. What I've found is...

- AI cannot write stories on its own. It just can't, full stop. Some people try, including me, but the results are nonsensical without significant tweaking. I expect that to change eventually, but not without a conceptual breakthrough or two.

- AI is immensely useful as a prosthetic imagination.

What I use it for isn't to write the story for me. It's to, in case I ever get stuck at some point, offer me suggestions for how the story can continue -- suggestions that I can accept or deny. Even if I deny it, it's useful as a way of illuminating my own ideas for the story. There's got to be a reason I don't like that continuation, and that is often enough to think of something I do like.

In other words, it's mostly eliminated writer's block.

It's also handy for expanding my vocabulary. English is my third language, and while I like to think I'm good enough for daily life -- I've lived in Ireland for over a decade, after all -- there's a big difference between 'good enough for daily life' and 'good enough to write good fiction'. Prior to using NovelAI, my writing was... dry. Conceptually heavy SF doesn't necessarily require high-end wordcrafting, but it helps.

The AI, especially when told to emulate Sheridan Le Fanu or any of the other great authors, is better than me at this. And since I can ask it to jump in at any point, it's become the most attentive, capable cowriter I've ever had. Perhaps noticing this, NovelAI now calls their default AI tuning 'Co-Writer'.

It's still likely to write something I can't immediately use, but that just means I need to absorb its ideas and make them my own. Repeat a hundred times per day, and I end up learning much, much faster than I ever did when I was writing on my own.

To summarize, I don't use AI to write my stories for me. I use it to get better at writing.

I think it should be possible to do the same for other forms of art.

AStrangeMorrow• 5 years ago

If interested in an other project that offers good tweaking, artbreeder (artbreeder.com) has been around for a while too. You have different models to play with (faces, anime, general, buildings, paintings etc...). Not all of them have the same level of quality and customization, but for example if you go for the face one, you get a really high level of customization: "cross-breeding" of 1 to 8 faces and then tweaking of many attributes (gender, color theme, age, ethnicity, realism, and other various details).

deft• 5 years ago

I made a waifu, then I pressed save. It wanted me to signup so I pressed back so I could just rightclick saveas but... I lost my waifu forever now :(

jimmygrapes• 5 years ago

This is like the condensed story of humanity

GoblinSlayer5• 5 years ago

Waifu is kept in memory, normies.

tsukikage• 5 years ago

"Keep precious things inside you, or you will lose them"

twic• 5 years ago

It's okay, Step 43636 will come to console you in your dreams.

keewee7• 5 years ago

This is why we need local self-hosted AI. Keep the waifus safe.

jcun41282• 5 years ago

Could have been an NFT

satronaut• 5 years ago

comment of the year, and it's not even friday

ASalazarMX• 5 years ago

I mourn for your lost perfect waifu, but if it helps, your comment likely saved other waifus because we clicked [Download] instead of [Save].

echelon3• 5 years ago

This is mind-blowingly good. You keep pushing the state of the art further to the point of broad applicability. It won't be long until everyone can be an artist without putting in the ten thousand hours of drudgery of training their muscles, hand-eye coordination, structure of shape and perspective, etc. I can't wait!

Do you have a team page? How many of you are there? Do you work with gwern and nearcyan? Are you going to raise for this? (You should totally scale this!)

Great work, and keep it up!

Cixelyn6• 5 years ago

No team page yet! Friends with near, but haven't had a chance to meet gwern yet (maybe one day??)

Arrowmancer is really our first attempt at scaling it up; hoping to do even cooler generative AI-related in that production.

echelon3• 5 years ago

All the best! Super excited!

dirtyid• 5 years ago

I hope these generators expands into non waifu / pretty boy anime depictions. There's a lot anime gaijin faces out there to explore.

aniforprez3• 5 years ago

I would like to see how it generates late-80s/early-90s style features. The current pool of anime art styles are very generic (aside stylistic outliers) and I'd love to see Cowboy Bebop/Akira/Bubblegum Crisis type character designs

echelon3• 5 years ago

Miyazaki's style of character design and fantasy would be amazing.

liuru17• 5 years ago

This one has some rough-looking ones! They're a bit rarer than most.

bmitc6• 5 years ago

This isn't really creativity is it? I like to call stuff like this statistical copying, and indeed, the linked Wikipedia article on GANs says:

Given a training set, this technique learns to generate new data with the same statistics as the training set.

There isn't a creative process here nor any creative introspection going on. While the technical results are impressive, this article does not address creativity even superficially, and just slaps the label on. There isn't any AI either. It's machine learning, i.e., statistical models and algorithms.

aniforprez3• 5 years ago

"create" doesn't necessarily imply "creativity" so I'm not sure why this is brought up. Of course generated images right now don't have any creativity attached to them and that's not the purpose of this article. All the images shown seem to mimic the generic "anime" art style and seems close enough to what a human draws which seems to have been the goal

zaik• 5 years ago

There isn't any AI either. It's machine learning, i.e., statistical models and algorithms.

I always thaught those were synonyms.

xcambar2• 5 years ago

Both the A and the I in AI are debatable terms.

The sole definition of intelligence is under heavy load of reconsideration the last decades with the emergence of a better knowledge of animal cognition, for example.

isaacimagine• 5 years ago

There isn't any I either. It's biological learning, i.e. neuronal models and feedback loops.

/s?

Gigachad8• 5 years ago

Purists seem to define AI as anything that is currently too hard for computers. Wasn't long ago that chess engines with no neural networks or machine learning were being called AIs.

bmitc6• 5 years ago

AI has been co-opted as a synonym for machine learning, if only for marketing purposes. Machine learning existed for decades before it was "renamed" as AI.

I just don't see anything intelligent yet. People somehow have gotten confused with the success of machine learning being treated as intelligence. We have a lot of statistical models of things that are very successful, but those aren't considered intelligent. For reason, machine learning telling you something about data has suddenly been treated as AI. Machine learning can do some impressive things, but I think its short-sighted to equate AI and machine learning.

Intelligence is really a tough thing. Watching a video of even a single-cell organism displays a sort of intelligence and behavior far beyond anything I've seen of machine learning. So why is it intelligent? Or is it intelligent? I'm not entirely sure, but my point is that machine learning is orders of magnitude incapable of describing (i.e., modeling) even the simplest self-directed and self-adapting behavior that we see in the real world.

numpad08• 5 years ago

I find odd that no one is tracing back portions of generative arts back to publicly posted sources. Internet horde of bored people were used to be good at finding where an unattributed pieces come from. Are these really generalized and stored in highly abstract forms? I doubt it...

plutonorm4• 5 years ago

What makes you think human creativity is anything more than statistical modelling? What else can it be, based as it is in a physical substrate? Objections like yours are usually a sneaky way of implying that human consciousness breaks free of the physical world, cartesian dualism hidden within people's arguments.

"It cannot be creative because it's only bits/cogs/linear algebra/etc/etc." Well describe to me the way it is different to the processes of the human brain? "There is some magic sauce in the human brain we do not yet understand!", well then how do you know that this magic sauce does not exist within the statistical models inside the computer?

I find it very irritating that such shallow reasoning prevails amongst intelligent people.

omnicognate3• 5 years ago

What else can it be, based as it is in a physical substrate?

This is no less shallow reasoning. The question of whether the academic field of statistical modelling already contains the necessary ideas to produce strong AI is not decided, and won't be unless/until somebody makes a strong AI. People have different intuitions about what the answer will be and until it can be determined empirically I suggest treating them as what they are: intuitions.

capableweb2• 5 years ago

What makes you think human creativity is anything more than statistical modelling?

Our ability to make the decision between following the rules and breaking the rules, when suitable. A computer could also break the rules, but in most cases it wouldn't make sense or look like good, while a human could make judgement about when to break the rule. Sure, we all learn by copying, but after a while, we start getting a feel for when to break the rule, and that's when unique art appears. Computers seems to not have learned this fact yet (or rather, haven't been taught that yet).

Using the tool that this submission offers, all the results will look similar and can be traced back to the training set you give it. Do something similar with a human (over similar amount of time that the machine got, in terms of human time) and eventually the results will look way different than the training set, as what we see with artists in real life.

YeGoblynQueenne4• 5 years ago

> What makes you think human creativity is anything more than statistical modelling?

For example, humans don't need to see millions of examples of waifu before they can draw their own.

Also, humans can draw in different styles, including novel styles that look nothing like styles they have seen before. Statistical models like GANs can only draw in styles similar to the ones in their training sets.

Statistical modelling can only represent the data in a training dataset and is incapable of novelty. Humans are capable of novelty.

> Well describe to me the way it is different to the processes of the human brain?

We haven't created the human brain and it's very unlikely it uses a technology we understand, like linear algebra.

bmitc6• 5 years ago

You imply a bunch of things I didn't say. We don't have a definition or model of creativity or intelligence, even within humans and other animals. Most humans struggle to view other things as intelligent when they are at a close inspection. We have some mild understanding at a sort of system level, but it's usually enough to generally bin things up into not intelligent or not.

For something like the original post, we do know what these things are. They're statistical models. Full stop. They show no indication of what we see in creative and intelligent behavior, that is the ability to self-adapt to both internal and external initiatives. This GAN in the post has no ability to step outside of the statistics in the training set unless the model is updated to prod it to do so. The model can be changed, but it is a forceful change. If you show me tens of thousands of images, I am not, at an emergent, top-level, system level, etc., bounded to the statistics of that image set. Is this GAN asked something or given a goal aside from an implicit "draw something like what we've given you"? Even if I do draw something like or akin to the given image set, I have full creative control over the image (assuming some drawing skill).

If the human brain (and really body) can be modeled via a statistical model (which is not yet known but is surmised as you imply), that doesn't necessarily explain high-level behaviors. More is different. You call it magic sauce, but others call it emergence. Our understanding of emergent behavior and complex systems at large is still in work.

In my view, metaphorical thinking, of which analogical thinking is a subset, is a likely kernel of human intelligence. While these statistical models are copying, which is similar in a way to analogy building, it's not quite there. The reason things it generates looks like other things is because it searched a parameter space for matching statistics. However, it cannot even explain that's why it generated what it did. We explain for it. These things are no more artificially intelligent than things like thermodynamics are naturally intelligent.

Lastly, as I pointed out in my original comment, if this is indeed creative as someone like you implies, the article fails to make a convincing argument and bounces around a lot of buzzwords.

I find it very irritating that such shallow reasoning prevails amongst intelligent people.

I was offended, but I suppose I agree. ;)

liuru17• 5 years ago

read the bottom! The part about creativity is on the bottom :D

noobermin• 5 years ago

One of the things being an anime fan has made me realize is how the industry goes through phases in everything: gimmicks, art styles, tropes, etc. This can at best replicate what exists, but it cannot create new styles which honestly comes from the creative process of hundreds to thousands of artists creating and influencing one another.

People keep forgetting that you really can only fit to data you have. Extrapolation exists as a concept but the requisite intuitive knowledge needed in order to create something new that can be successful is hard to understand even by humans at this point (how much of the business press is full of gimmicky blogposts about how to be successful, full of contradictory anecdata, opinions, advices), I don't even know how AI researchers would go about tackling that. I am not an AI person but as a plain old scientist I know extrapolation without intuition is almost always a fraught effort.

bmitc6• 5 years ago

but it cannot create new styles

Sure it can, but then it's not considered anime anymore. I think this sentiment is confusing genre with training set constraints. A new model is not required for some artist to do something different from anime or even an anime artist to do something different. Humans can self-adjust all with approximately the same base model (whatever that is).

commandlinefan• 5 years ago

This isn't really creativity is it? I like to call stuff like this statistical copying

Or, through decades of AI research, we're just now starting to better understand what actual creativity really "is"?

aimor• 5 years ago

What methods are there to estimate how many unique characters a model can generate? The answer is not infinitely many, but determining when two images are of different 'characters' is fuzzy.

gwern• 5 years ago

It's hard to say, but I think a useful measure would be to look at mode-dropping compared to the training data. Whatever the 'number of unique characters is', it clearly ought to be at least as large as the characters you see in the original training data, right?

For TADNE, Arfafax ran Danbooru2019 and a few million TADNE samples through CLIP to get the image embeddings, and clustered them; when the two sets of clusters were graphed using tsne, you could see that the TADNE StyleGAN2-ext did a lot of mode-dropping in that many smaller outlying clusters of characters/franchises/topics simply did not appear in TADNE samples. The TADNE looked like a big galaxy, while Danbooru2019 looked more like it was surrounded by archipelagos. TADNE was extensively trained on them and was a very large model, but the GAN dynamics & StyleGAN architecture mean it didn't do a good job absorbing rarer/more idiosyncratic Danbooru2019 image-clusters.

I expect newer generative models which avoid GAN losses and which use more flexible (but expensive!) architectures, like DALL-E, would perform much better in terms of mode-dropping, so you'd see a lot more unique characters/images out of them. (I'm very excited about them. As good as TADNE or Waifu Labs v2 may be, I think they are still far behind what could be done with just existing data/arch/compute.)

tasha0663• 5 years ago

Step 43636: During this phase, the training gets unstable at times, so we have snapshots of occasional horrors like this.

Ah, make that three things the public shouldn't see being made: sausage, legislation, and waifus.

jakey_bakey• 5 years ago

I wish I had the stones to call my company Waifu Labs

SavantIdiot• 5 years ago

I didn't know what you meant so I googled it. The technology is cool, but the content is ... problematic.

From urban dictionary:

   "Waifu" is used to refer to a fictional girl or woman (usually in Anime, Manga, or video-games) that you have sexual attraction to, and you would even marry.

Huh.

kadokaelan2• 5 years ago

youd be surprised at the waifu market size

harpersealtako• 5 years ago

A big part of it you're missing is that it's a joke. Anime fans definitely know they're weird, and are very passionate about the things they like (and are conscious of that), and thus a lot of humor in the community is self-deprecating and ironic (for example, calling your favorite fictional character your "waifu" or "husbando"). The fact that outside observers might think it's "problematic" is kind of the whole point.

akomtu• 5 years ago

This tool has many applications, and those that will make you rich isn't about anime.

peoplefromibiza2• 5 years ago

step 1: generate random waifu

step 2: NFT all the things

step 3: profit

step 4: GOTO step 1

step 5: automate steps 1 to 4

sandos• 5 years ago

Oh, I see what you are saying....

More things for, like, adults?

scollet• 5 years ago

It needs to make money?

w_t_payne• 5 years ago

I found the most interesting part was the evocative comment about the 'vast and parched' nature of the latent space.

I wonder if the OP's intuition regarding the sparseness of the latent space, and the relatively small area occupied by the 'useful' manifold? embedded within it provide us any clues as to what symbol grounding might look like for some neuro-symbolic infrastructure that sits atop that latent space.

I.e. how should we be trying to represent concepts like 'male' and 'female' within that space?

Is it important to have these concepts represented as a low dimensional manifold?

Is it important that this manifold be easily described by some simple geometric form like a convex polytope?

Is it important that nuances and variations on the concept be separable within the bounds of the concept-specific manifold?

What other properties might be important?

Kuinox• 5 years ago

It's funny how people complained about github copilot 'stealing' people code, but nobody here complained about this AI 'stealing' artworks.

Don't get me wrong, I have nothing against this, but I think we should start discusing morality of AI generated content, even if it doesn't train on existing artworks/code.

7373737373• 5 years ago

Have you considered applying similar models to VR avatar creation? That's a market in itself

Tade0• 5 years ago

Amazing work and progress. The previous version appears almost toyish in comparison.

FYI uBlock Origin complains about the registration link, because it on "Peter Lowe’s Ad and tracking server list".

pgl• 5 years ago

That's because it bounces the registration link through a tracker.

If you're OK with being tracked, you can permanently allow that domain.

jcun41282• 5 years ago

I would be interested to find one I like and use it for a desktop companion project, not an original idea but I am not an artist ha.

syntheweave• 5 years ago

The image quality is good, but now I realize I'm experiencing "uncanny waifu". Authentic character designs bear two things in common:

1. Simplifications of reality(the actual artist training method would be traditional studies off life and photo reference followed by gradual reduction and symbolization to a style)

2. Symbolic meaning. Things like the style of eyes, clothing, etc are all meant to signal personality. This is stuff that current AI techniques don't really touch upon in any direct sense.

Since the ML method is built on interpolating off final results, it's going to lack in these qualities and produce something that is consistently an "average impression". Akin to asking the algorithm to generate mythical heroes by mashing up the various stories: you get a hero that is somehow the average of Icarus, Heracles and Achilles, which would be less of a character than the originals.

someone7x• 5 years ago

Could it work backwards? Eg, to take a hero like Heracles and determine he's say 40% shared with Gilgamesh. Then we might see that even the originals aren't very unique.

Just a thought, I don't really know anything about ML.

rambojohnson• 5 years ago

project name is kinda incel cringe :(

dmix• 5 years ago

So putting some Anime digital illustrators out of work? ....or I could see many simply use them and pretend they did it themselves?

Obvioisly there will be plenty of illustrators doing custom work that these can't (yet) replicate.

Also good for those countless anime avatar'd Twitter users.

ausbah• 5 years ago

any good info on manipulating the "control vectors" in the latent space?

easrng• 5 years ago

Cool, it's like an AI-generated picrew

Welcome to Waifu Labs v2: How Do AIs Create?

Related Stories