Greg Rutkowski was removed from Stable Diffusion; AI artists brought him back

https://decrypt.co/150575/greg-rutkowski-removed-from-stable-diffusion-but-brought-back-by-ai-artists

by ecliptik • 3 years ago

59 58 3 years ago

While Lykon maintains he'll readily concede to Rutkowski, he said he believes he is acting on behalf of the greater good.

Sounds pretty unethical. :(

watchtower• 3 years ago

Yup, that's pretty much it. Followed by the next sentence:

“If he contacts me asking for removal, I'll remove this.” Lykon said.

... as if Greg hadn't taken a clear stance.

jeroenhd• 3 years ago

But he didn't specifically tell me to stop!

There's no stopping this technology, but we can do without all of fake "for the good of humanity" stuff. If you want free pictures in someone's style without paying an artist for them, just be honest.

lolinder• 3 years ago

Others sided with Lykon, alluding to the fact that Rutkowski's art has already been in circulation in SD1.5 for years.

Fun fact: we have yet to reach the one year anniversary of the initial announcement of Stable Diffusion [0]. A lot has happened in one year!

[0] https://stability.ai/blog/stable-diffusion-announcement

Avisite• 3 years ago

I really dont think using his name in a prompt had as much to do with his style to some degree, but more that it had an unusual amount of influence on the resulting image compared to other artists. Like his name had an unusual weight to the model that made images higher quality. Images that are nowhere close to his style still had "by Greg Rutkowski" in it. in the same way they have "Masterpiece, Best Quality"

jsheard2• 3 years ago

If Rutkowski didn't already exist, would AI artists ever be able to arrive at his style from first principles? It seems that all novelty entering the AI art ecosystem comes from riding the coattails of non-AI artists, they're literally incapable of producing, for example, fan-art of a newly revealed character until non-AI artists produce enough examples to train a LoRA on.

Months on from the release of the new Zelda game I still can't find any on-model AI art of most of the distinctive characters. They just can't do it without a LoRA, and you can't make a LoRA without many existing examples, so they never could have put those characters to paper using AI if the design only existed in their head. Conventional artists have to do it first so that AI artists can follow.

bugglebeetle• 3 years ago

Artists using AI maybe could have. “AI artists” are largely a bunch of obnoxious trolls with a shallow understanding of art history, technique, etc, such that this seems unlikely. Almost all AI art I see is within a narrow range of stylized realism or anime/manga-derived work.

brucethemoose2• 3 years ago

There are novel artists using Stable Diffusion or Firefly as a photoshop-like-tool. Not txt2img, but stuff like inpainting, controlnet, contextual modification, LORAs trained on their own portfolio and such.

They are obscure for the same reason most artists are obscure.

But most "AI artists" you are talking about are would-be consumers looking for something specific, not creators looking to pull something novel out of their head.

dale_glass6• 3 years ago

Probably something similar? I don't see why not.

I don't look at that kind of artwork all that much, but Rutkowski's pictures seem to have a very D&D/Tolkien-ish quality to me, to put it in some words. I'm pretty sure other artists made works in similar styles.

I do recall purchasing a poster with a similar look around 20 years ago, when he'd have been 14-ish. He might be a prominent artist in the genre now, but he's far from the first or that unique.

lfuller• 3 years ago

Greg Rutkowski is most well known for his work on D&D and Magic the Gathering. You recognize his style as he was one of its pioneers.

CuriouslyC3• 3 years ago

Putting greg rutkowski in a prompt produces compositions that have a fantasy/sci fi edge, more muted earth tones, areas of both clean and rough brush strokes and areas of large contrast with a tendency towards a central focus. It also makes for better dragons/castles/etc. Nothing magic.

cheald4• 3 years ago

By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different?

A fan artist must observe a character's official depiction before they can produce their own variants of it. Most humans do not produce art in a clean room, either - we all produce a synthesis of the existing art we've been exposed to. A LoRA or dreambooth model incorporating a novel concept is conceptually similar to a fan artist being shown a new character so they can produce new derivations of it.

You could produce "police sketches" of a previously unseen character based on shared description, but it would likely fail to capture the nuance of the individual just as police sketches would.

jsheard2• 3 years ago

By the same token, you had no ability to conceive of any original character before it was shown to you; why would an AI art model be any different?

I've yet to see a LoRA produce a meaningfully diverse and high quality range of interpretations based on just a single piece of character key art, or brief appearance in a trailer, the way the conventional fan-artists do. What actually happens is that those conventional artists produce a wide variety of interpretations, someone scrapes those hundreds or thousands of images, and then trains a model using that dataset instead. The model needs the variety and novelty that the human artists bring to the table, otherwise it would just over-fit to the official art.

When I was looking for examples of Zelda AI art I noticed there is a LoRA for Purah, and the first version was posted on CivetAI not when her TOTK design was revealed, nor when the game came out, but a month later when there was a wide enough corpus of existing fanart to scrape together a decent model with. It only works by piggybacking off the non-AI artists effort.

astrange• 3 years ago

Rutkowski was /never/ in Stable Diffusion - the article is incorrect.

His name worked in the text encoder released by OpenAI, but it was more like a synonym for "concept artist". The image encoder didn't see any of his images.

So if he's not in a newer text encoder, you could find him again with textual inversion.

neilv• 3 years ago

“If he contacts me asking for removal, I'll remove this.” Lykon said.

My interpretation of the article is that the entire reason this person created this was because the artist had objected to being in a generative model. When the artist was removed from the popular generative model, this person created a model/tuning specifically to put the artist back into one, and distributed it. They seemed to do this well aware that the artist didn't want to be in a generative model.

That sounds aggressively adversarial and abusive, and so the quote above sounds disingenuous.

Consider the analogy of a bully who is hitting a victim, and the bully demands that the victim ask the bully nicely to please stop. Humiliating the victim, and making the victim acknowledge the bully's power over them. Also, this is only after much damage is already done to the victim. And, in this case, after the bully has called over a mob to also beat the victim, and knowing that, even if the bully stops, the mob will continue.

The person in this case might not have the self-image of a bully, but is instead thinking of it as the usual Internet get-out-of-jail-free card: such as when you post someone else's content, when you're pretty sure they don't want you to, and without even trying to get permission, but you think it's all OK if you say "I'll take this down if they ask me to". (Related: The YouTube copyright violation disclaimer "I don't own this! All rights to the owner!", maybe even referencing the "All rights reserved" of a traditional copyright notice.)

Wherever you stand philosophically on copyright and related matters, and whatever jurisdiction and economic conditions you're under, it's maybe also helpful to consider the bullying question. One can stray into that accidentally, and that's harder to justify.

arjun_krishna1• 3 years ago

based

wccrawford2• 3 years ago

If this person was genuine, they would have asked permission first. It wasn't like they had to ask a lot of people who are hard to contact. Just 1 person.

I'd have had more respect for this person if they'd just said "Sorry not sorry." instead.

olaulaja2• 3 years ago

The thing is, most styles don't have names, only artists using them. If you remove artist names from the training data, you lose the ability to describe those styles, including as combinations (so you can't even do "influenced by rutkowski" styles). This obviously sucks for the user, and I don't think artists should be able to deny control over style just because it makes imitation easy.

In this context, putting an artist back in the model is a pretty sensible form of disagreement/protest. I admit the "I'll take this down if they ask me to" is strange, but mostly because its too weak; the LoRA should just stay up, not demand another no from someone who already made their opinion clear.

caconym_8• 3 years ago

most styles don't have names

If you want to educate yourself on how to refer to different art styles you can start here: https://en.wikipedia.org/wiki/List_of_art_movements

If you remove artist names from the training data, you lose the ability to describe those styles

It's almost as if artists develop their own styles that are just as uniquely theirs as any one work produced by them in that style, if not more so! When people put "by Greg Rutkowski" in their AI art prompts, they are not asking for results in a style that exists independently of Greg Rutkowski despite his name somehow being the only way to identify it. They are, in fact, asking for results in the style of Greg Rutkowski in the sense that works by Greg Rutkowski were labeled as such and fed into the training process, programmatically creating a digest of works by (at least) Greg Rutkowski that we call a "model", and that model is as a result capable of mimicking the style of Greg Rutkowski specifically. They are asking for results that look like they were painted by Greg Rutkowski, and they care enough to ask for Greg Rutkowski specifically because Greg Rutkowski is a highly skilled artist who developed his own distinctive style which is the central pillar of his intellectual property as an artist, and his competitive advantage.

Intellectual property law as it stands today obviously couldn't have anticipated this technology, and didn't. However, if this kind of thing is still legal in ten years, that will represent a major systemic failure of copyright and trade secrets law to protect the core intellectual property of artists in the same spirit as it currently protects intellectual property in other domains.

dragonwriter4• 3 years ago

If you remove artist names from the training data, you lose the ability to describe those styles

Do you?

With base SD1.5, “by greg rutkowski” produces extremely similar results to “detailed realistic muted fantasy”. Except for the case where what you are actually trying to do is directly mimic the work of an artist with a strong unique style that is very heavily represented in the source data (Rutkowski, IIRC, has extremely little representation in the SD training data, leaving aside question of whether the other parts of that description apply), I don’t think you are losing the ability to refer to styles without artist names.

thulle• 3 years ago

I think Civitai, where the Rutkowski LoRa is hosted and where the screenshot in the article is from, in part set the tone during the earlier conflict with Sam Yang/@samdoesarts. I think Yang posted about someone on r/StableDiffusion making a LoRa trained on his work, which prompted Civitai to host a contest to see who could make the best samdoesarts-model. And send him a mail: https://files.catbox.moe/adyb0x.png

cateye• 3 years ago

Greg Rutkowski's art style is similar to Gustave Moreau's, Rutkowski shouldn't be permitted to create his artwork.

throwaway69123• 3 years ago

There are too many artists applying human stylegan to their artwork! Something must be done to stop this unethical practice.

causality0• 3 years ago

A digital quagmire where Rutkowski himself struggled to differentiate between AI-generated pieces and his genuine works.

Is this hyperbole or did he actually say this? How could someone spend tens of hours meticulously painting something and then not be able to tell if he did it?

drdaeman2• 3 years ago

And if they're somehow (just theoretically) genuinely indistinguishable even to the author - this prompts to question the value of originality.

cheald4• 3 years ago

Nonsense. A neural net of sufficiently high parameter count can learn a function to reproduce any digital representation of information. This argument would suggest that anything which can be represented digitally is therefore of minimal value.

dale_glass6• 3 years ago

I suppose that means that he's having a hard time explaining to how to distinguish the AI version from his works. Like there's nothing along the lines of "This is clearly not my work because I don't draw dragon wings this way" he can say.

Not that he forgot what he drew and what he didn't.

Modified3019• 3 years ago

That reads to me as "he looked at the details of the AI art, and had a hard time finding obvious issue in the 'technique'", not that he forgot about art he made.

vitorgrs• 3 years ago

Weirdly this is like the best compliment I ever heard to AI Art.

Loveaway• 3 years ago

Man I feel for the artists. Seen a lot of outcry among people I've worked with too. It just seems so unfair. Like what are we doing? We automated away one of the most pleasing, satisfing and skill intensive endeavours humans can undertake. A dreadful reality we'll probably have coming to our jobs soon enough too.

Bitter truth is though, it's not going away. The genie is out of the bottle, and if you are a talented artist, you absolutly need to embrace AI. If you take a look at DeviantArt AI section for example you'll see: The bar has been raised. It's masterpiece next to masterpiece now. Talented artists + AI = Simply stunning work.

It feels heartless, but what can we do, will you be able to stay competive without AI? It feels you'll be just hampering yourself for idealistics reasons.

CuriouslyC3• 3 years ago

We didn't automate away art. We automated away the technical skill of digital illustration, which a totally different thing. Now, instead of artists also having to be illustrators, we've opened up being an artist to a wider audience.

Gigachad• 3 years ago

People know this but it doesn’t make things better. If you dedicated thousands of hours to mastering technical skill of illustration, it isn’t nice to see the value in that work obliterated overnight.

What makes this so much worse than things like chess AI is that chess is done for fun and the final output is useless. While art is mostly for the output. While the process is fun, it’s fun because of the value of the output.

caconym_8• 3 years ago

we've opened up being an artist to a wider audience.

No, we've opened up the ability for someone who writes a short textual description of an illustration to obtain an illustration ostensibly matching that description for a much lower cost than they might have paid for it before these technologies were available, e.g. by commissioning an artist.

You're right that technique can be abstractly separated from other aspects of the artistic process, but you miss how the inability to guide the application of technique will cut your capacity for expression off at the knees. The prompt is the art, such as it is, and the ability to iterate on versions of the prompt and parameters of the model to random-walk your way to a result that approximates what you were imagining is a very poor substitute for the ability to lay down the brush strokes yourself.

wccrawford2• 3 years ago

While I do think many of them are stunning, there are quite a few that are obviously AI-generated with almost no human interference... Or perhaps none at all. The scenes don't make sense visually.

I think to be commercially competitive, you need to embrace AI. But to just do good art, you can still get there the old-fashioned way. It's similar to CNCs in that you can get a computer to do it for you, but you many, many people still do it completely by hand. And some people go half-and-half.

ForestCritter2• 3 years ago

The reality is that the really good artists will stop feeding their art into the internet and that will be everybodies loss.

vasco• 3 years ago

Interestingly enough, Rutkowski might have been the biggest artist beneficiary of this AI hype cycle.

Aerroon• 3 years ago

The only reason I know he exists is because of SD. The same goes for a few other artists. Now I even know roughly what his art like like.

dkjaudyeqooe• 3 years ago

"Legal doesn't always mean ethical."

Right now it's not given that it's legal. Most people are assuming that it's ok under fair use, but until the SCOTUS rules on it it's an unknown.

Given the potential economic impact on the original artist, the mechanical nature of the transformation and the fact that extracting the style of an artist's work could be deemed substantive, fair use is not in the least bit a slam dunk.

One could argue that use of this sort of AI robs the original artist of the fruits of their labour, neutering copyright protections, and produces works that substitute for the originals, without adding any creative input other than commanding the output from a machine. Given the intent of copyright law, judges would likely be receptive to this argument.

Greg Rutkowski was removed from Stable Diffusion; AI artists brought him back

Related Stories