I am frustrated with Stable Diffusion

https://novalis.org/blog/2022-12-05-i-am-frustrated-with-stable-diffusion.html

by luu • 4 years ago

172 116 4 years ago

jarbus• 4 years ago

Online, stable diffusion looks incredible. But it's been so, so bad for me, and when you look closely at lots of other "good" pictures, they have lots of flaws too, we just aren't good at picking them out. I wonder if there will be a way to make SD align with what humans want more, maybe with some GAN approach, without needed actual "understanding" of the real world.

dwringer• 4 years ago

Using img2img and making subtle alterations/substitutions to the prompts and/or images as you go can allow one to serve the adversarial role oneself as the user. Ultimately it's sort of like sitting across the table from an artist who can't speak but you can pass drawings back and forth iterating on them until they are complete. Of course it can be drawings, 3D animation style rendering, painting, photorealistic renders, etc.

kecupochren3• 4 years ago

Try MidJourney instead. Very little is needed to get amazing results. You just set the theme and let it run wild. Or you can also be very specific and it will handle it.

skybrian• 4 years ago

I tried this using Stable Diffusion v2.1 at dreamstudio.ai. I'm not getting good results yet.

It seems like an odd place to build a dock, if you can avoid it. Is there a real-world place like that?

paulcole• 4 years ago

This place in Malta is pretty close:

https://www.alamy.com/stock-photo-boats-at-dock-near-a-set-o...

I actually thought the first version in the blog post was pretty good. It looked like a floating dock to me (that goes up and down with the water level) rather than one secured by pilings.

robocat• 4 years ago

Cap de Formentor in Majorca is an example, and I'm sure you could find better examples for other lighthouses. I remember clambering down the epic walkway to where all the lighthouse building materials were brought in - and there must have been a jetty or something because it's fucking steep. The best description and image I could find: https://www.majorcadailybulletin.com/news/local/2021/03/16/8...

I think that the "cove" and walkway is shown on the left of the third photo in the article.

That said, any jetty would have had to be cantilevered from the rockface. The guy in the article drawing piles next to a shear cliff of rock is hardly thinking the engineering though!

humanistbot• 4 years ago

I had the exact same experience. I foolishly paid for a pro account without even trying out the free version, because I was so excited by it. Only to learn that it took so much trial and error --- sorry, "prompt engineering" --- to get anything remotely usable.

This is the same with all kinds of generative models. Humans are a critical part of the system. If this is "artificial intelligence" it only appears intelligent because there is a human gatekeeper.

yunwal• 4 years ago

All AI models so far are pretty much only useful as a middle visualization step in a creative process. You have to come up with a vague idea of what to draw, SD can give you a number of compositions to choose from and will get close enough that you can tell the final product would be good. You will still have to recreate or heavily edit whatever it creates though. It’s still useful, but it’s not good enough to be a revolutionary product yet or declare photoshop obsolete.

Similarly, ChatGPT is a great replacement for a search engine. I can ask it to guide me through writing a metric to Prometheus from a batch job and it’ll dig out the relevant pieces from the documentation and present them to me in an easy-to-understand format. But I’m not giving it write access to main yet, or letting it make business decisions.

Slow_Hand2• 4 years ago

This sounds like any other tool or instrument that requires practice to render good results. SD is magic, but it's not a miracle. It still requires work and understanding to get good results. It's just a different kind of work than we're used to doing. Don't expect it read your mind. It's still a computer.

As a client, if I hire an illustrator to commission a specific image I'm not just hiring them for their technical ability to render images. The best illustrators are going to have a broad and deep awareness of all styles, modes of presentation, and compositional techniques. They're going to use all of their experience and knowledge to interpret what my needs are and deliver on that. That's not a trivial thing to do.

Even the best contractors have to deal with clients who don't know how to ask for what they want. The "I'll know it when I see it" type of client. As the "client" in this situation, you need to know how to ask for what you want in a way that your illustrator can interpret.

Fricken3• 4 years ago

When I was working as an editorial illustrator, the goal was to have a distinctive trademark visual style, so that when art directors commissioned something, they had a reasonable idea of what they could expect to get. If they wanted something in a different style, they would find an illustrator who specializes in it and hire them.

Last week I was telling my artist friends that, however upsetting this technology is, the game has changed. You can't put the genie back in the bottle.

I realized last night however that that is incorrect. Whatever "genie" artists have spent their lifetimes nurturing can now trivially be captured and confined to a bottle.

2devnull• 4 years ago

Maybe you can use gpt to automate prompt engineering and crowd source the image selection process then you can just let it run while you do other stuff?

I’m pretty sure that some of the people posting chatGPT on Twitter are using Twitter to crowdsource the output review process. In other words script the loop from generation to Twitter post and use upvotes to decide what output is most interesting. Been thinking about doing this myself but I don’t use Twitter much so no users to crowdsource from.

ekidd2• 4 years ago

This is the same with all kinds of generative models. Humans are a critical part of the system.

Part of what I find frustrating about Stable Diffusion (and to a lesser extent Dall·E) is that they're so ridiculously far behind ChatGPT in some ways.

For example, I ask ChatGPT (in French) for a short story of three firefighters in the style of The Three Musketeers. And it writes me a perfectly serviceable little story about a brave and heroic rescue, with good pacing and structure. And it's written using the French "passé simple" tense, which is normal for fiction but virtually never used in speech. And I get this in a second or two, on my first try.

Meanwhile, I ask Stable Diffusion for a picture of "an astronaut on a horse" (the demo prompt in my copy). And I get endless badly drawn astronauts on dismembered horses. Dall·E does a bit better, but I'd obviously have to generate dozens of images and accept something visually interesting that doesn't bear any resemblance to what I envisioned. Apparently I can do better with a lot of prompt engineering. But it's a lot more work than ChatGPT.

potatoman22• 4 years ago

SD's text embeddings aren't nearly as advanced as GPT3, so this makes sense. It'll get there; multi-modal models are only getting better.

CuriouslyC• 4 years ago

The scale difference between text and pixels is relevant. If you asked for a novel (less information content than a 512x512 image) from ChatGPT and judged it by how it held together at a high conceptual level relative to output from a stable diffusion image I don't think it would seem that different.

astrange3• 4 years ago

Stable Diffusion is a free 4GB model that can run on a phone. ChatGPT size is unknown, but around 350GB.

You can use a large text model to design prompts for a smaller image model if you want though.

macrolime• 4 years ago

Stable Diffusion is tiny compared to ChatGPT.

The largest image generation model I've seen images from is Google Parti, which is an order of magnitude larger than Stable Diffusion. Still small enough that you might be able to run it on a 24GB consumer GPU with some optimizations though, so hopefully Stable Diffusion will get there in some months.

Have a look at the four kangaroo pictures here to get an idea of the difference https://parti.research.google/

fy20• 4 years ago

What nobody tells you is that to get the results you see people post online often requires hours of work for a single image.

First you need to refine the prompt so it is ultra specific (which OP has not done), then you need to generate a hundred or so images and pick the best ones. From there you can use img2img to refine it more, and once that's done you might want to go to Photoshop to add some finishing touches.

To get good results it is an art form at the moment, but of course as the tools get better eventually it will just be a click of a button.

Fricken3• 4 years ago

So long as you're content with some random crazy image, you can get fancy stuff pretty easily with Midjourney V4. Getting specific details correct, however...

mch82• 4 years ago

Correct. It’s a new medium for a new school of artist, not a replacement for artists. Very exciting!

prpl• 4 years ago

it’s the new sketch

vhold• 4 years ago

Another trick you can do is when you find a composition you like, but it's off in whatever way, is lock down that seed, and then apply variation to that initial noise state, and produce a bunch more images from there.

The popular AUTOMATIC1111 webui implements it: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki...

godelski2• 4 years ago

As someone researching generative models, this is one of my pet peeves. There's been a rat race to show off high quality cherry-picked samples. Very few works actually show random samples. It is rather frustrating because when pointed out it still goes ignored. I'm not sure why we look the other way, aren't these issues we want to solve? But if you tried to publish a paper with random results you won't get accepted.

I am also not worried about AI taking over art, exactly for the reasons you say. Even if alignment was a whole lot better you're never going to be able to perfectly describe an image in your head. Language is too limited. So it will always be a tool. Of course, it can enable fraud. But not many people are putting digital arts on their walls.

The other pet peeve is that people think generative models are only image or text. They so so much more.

darawk• 4 years ago

You don't need to be able to perfectly describe something in your head. That's what you hire an artist for. Almost nobody hires an artist to produce the exact thing they are imagining - they have a general idea of what they want, and the artist works with them in a back and forth way to find something they like. These image models seem quite capable of doing that to me. You enter your vague description, and iterate from there until you get something you like.

orbital-decay• 4 years ago

you're never going to be able to perfectly describe an image in your head. Language is too limited.

Pretty much this - and that's why both "AI art panic" and "I won't need artists anymore as I can just prompt anything" (which OP seems to try) are based on the wrong premise.

In general, people tend to make 2 false assumptions:

1. That SD is a ready to use product (it's more of a "middleware" model that is designed to build products upon)

2. That text to image is all it can do, while the most power is in style transfer, finetuning, and the ability to be guided with higher order hints than just text alone.

What would be best right now to use it as a content creation tool, is to combine SD with several other models to have temporal stability and tagged-3D-to-2D, and to use some software toolkit with a pipeline like that:

- quickly layout a mock-up scene in 3D with rough assets, just like in game engine level editors, possibly tagging the geometry or objects with short descriptions, like "middle-aged man", "Volvo semi", "pine tree" etc. No need for detailed geometry, just stick figures and rough shapes.

- using one of the multiple available techniques, train your style on your reference images (which might either be curated output of the model itself, or a specific visual language you constructed)

- enter your prompt, which doesn't need to be complex and convoluted as you already provide the precise hints on what you want

- let the model render the result, based on the depth, tagged geometry, rough rendering providing shadows and reflections etc

a_bonobo• 4 years ago

'The Policeman's beard is half-constructed' is a collection of poems from 1984. They were generated by Hidden Markov Models, handpicked from thousands and slightly edited by humans. Not much has changed in those 38 years :)

sebzim4500• 4 years ago

What nobody tells you is that to get the results you see people post online often requires hours of work for a single image.

It is definitely a skill but I don't think it's true that good results generally require a lot of time once you know what you are doing.

Anecdotally I've seen requests on discord and 4chan answered within a few minutes with high quality images.

sirwhinesalot• 4 years ago

What (barely) works for me is:

- Make a prompt with a similar structure to the ones that generated good images online.

- Generate a bunch of low quality images.

- Pick the low quality image that kinda sort of has the shape and colors I'm looking for.

- Iterate on that image with various other prompts and img2img, increasing how much the original image contributes as it gets better.

theCrowing• 4 years ago

Mhm...looks like he put more effort into writing this blog post than learning how to work with SD.

InvisGhost• 4 years ago

I've had a lot of luck using Invoke.AI's solution. It allows for a lot more control over what the AI model is allowed to modify and lets you tailor each prompt for each individual edit.

xrd• 4 years ago

This is my favorite as well.

Are you running on your own hardware? I've found the latest commits to be less stable and it's hard to know what to run.

hackyhacky• 4 years ago

This should really be an easy fix, since Stable Diffusion is open source.

Just find and remove the code that reads:

  if (drawing_bottom_of_cliff)
     fuckup(dock, RIGHT_SIDE)

The team is waiting for your pull request!

Aeolun• 4 years ago

In my experience, Midjourney has been so much better than Stable Diffusion at anything game related that it’s absurd. It might be worth trying that.

anentropic2• 4 years ago

I've had exactly the same issues with midjourney, can't really tell any difference between then

kyleyeats• 4 years ago

It's all a composition and image-bashing game with Stable Diffusion.

A better strategy here would be keeping the same seed and then try adding 'dock' phrases to the prompt. Then image-bashing what it gives you over the first one. Image-bashing here basically means putting it in a new layer in GIMP over the old one and erasing the parts you don't want. It's really great with seed alternates because the two images will be very similar.

Inpainting is good for stuff you don't want the viewer to look at. If there's an errant shrub in your desert picture[0], you can inpaint it out. It almost never works for faces, heads, characters or any other focal object. The new depth stuff might fix it, but inpainting generally just doesn't have a good sense of how to orient things in the scene.

[0] I generated a bunch of SD images to spice up my CSS library's homepage: https://casscss.github.io/cass/

WheelsAtLarge• 4 years ago

I love SD for its unexpected results. You really don't know what you are going to get. Trying to get it to create the exact image you have in your mind is next to impossible. That's where this guy went wrong he thought he was getting an artist that would do as he asked but SD is not that. There's a lot of randomness associated with it.

elliottkember• 4 years ago

The gap between something totally new existing, and someone taking it for granted and saying it totally sucks, is still so short.

I'm reminded of Louis CK's bit about wifi on planes:

"And it's fast, and I'm watching YouTube clips. It's amaz--I'm on an airplane! And then it breaks down. And they apologize, the Internet's not working. And the guy next to me goes, 'This is bullshit.' I mean, how quickly does the world owe him something that he knew existed only 10 seconds ago?"

esprehn• 4 years ago

To be fair in the case of airplane wifi this is compounded by the exorbitant pricing.

If you pay a premium price you expect premium service.

There's no competition for airplane wifi though, you either use it or you don't. It's not like you choose between two different vendors... So they're competing against no wifi at all which allows for pretty mediocre and unreliable service as that's "better than nothing".

These free ML models exist in a slightly different world.

guiambros• 4 years ago

To be fair in the case of airplane wifi this is compounded by the exorbitant pricing.

Is this still true, though? I regularly pay $8 for wifi on a 6h flight coast-to-coast with United. And also paid $15 for a 10h transatlantic flight last week, which seems pretty fair to me.

Oh, and messaging and chat apps are free, so you don't really need to pay if you only want to keep in touch with folks on the ground. Sure, quality is sometimes spotty, but seems to have improved over the last couple of years.

totoglazer• 4 years ago

On my last flight it was $39.

coldtea• 4 years ago

If you pay a premium price you expect premium service.

Only the point of the Louis CK joke is that mere flying in the air, and getting to 1000s of miles away in a couple of hours, is a beyond premium service, when thought of in historical perspective... closer to a miracle.

It's this "taking for granted" that makes us consider it a standard thing in mere 100 years after its invention, and when it was not just a thing for rich people, but a totally unimaginable dream for millenia...

Choco31415• 4 years ago

I almost feel like that’s more of a reflection of someone’s attitude or even their mental health. Negativity can be nice, but insulting something for being what it is is not exactly intelligent.

joe_the_user• 4 years ago

The gap between something totally new existing, and someone taking it for granted and saying it totally sucks, is still so short.

I think it's the opposite. With any complex and challenging task, the gap between "mostly done in a mediocre way" and "finished and acceptable" tends to be huge. Lost of "mostly done in a mediocre way" Hollywood films never see the light of day 'cause there's no reason for them to. Lots of writers throw their "mostly done in a mediocre way" novels away and start again.

If this tool gets you to a place that seems "90% done" but you find you have do as much work for the remaining "10%" as if you would if you started from scratch, frustration seems reasonable.

ShamelessC• 4 years ago

It's satire - Just because you are now "justified" in getting frustrated, does not make it any less ironic.

krisoft3• 4 years ago

One super obvious trick, which the author seems to be not aware of:

If you have two pictures and you like one part from one and an other part from an other just photoshop/gimp them together.

If a particular manipulation is easier manualy there is no reason you need to try to convince the AI to do it. Just do it yourself.

In this case you drag the two images to two layers and use the erase tool to remove the blemishes which are present in your “top” image but not present in the “bottom” one.

joeld42• 4 years ago

You can only get so far with this, the lighting and perspective won't match and you'll end up with a "collage" look, especially as you add more images. You can sometimes feed the result back through img2img and get it to fix it. Even in an illustrative style like this the color tone is going to look different.

krisoft3• 4 years ago

Sure thing. But I mean it in a much more restrained way.

They have one picture where they like the left side of the dock, but say that the AI screwed up the right side. And they have a previous one where the right side is good but the left side is bad. And they are frustrated that the AI can’t or won’t make a single generation where both things are good.

And the two pictures have the same structure, same illumination and look barely distinguishable otherwise. That is the case where editing is trivial.

You might wonder, how often does this really happen in real life, and the answer is: all the time. You get a picture you like overall, and it looks good from 10 feet distance but on closer inspection you notice some blemishes with it. One way to fix this is to “inpaint” the blemishes. Basically you tell the model to keep all the pixels unchanged, but generate new ones in a small region masking out the blemishes. And of course you run many such inpainting generations and pick the best. And when you have multiple blemishes, which is often the case, then very often one gets filled in good in one generation while the other is better in an other. And you can keep rolling the dice hoping that some day the AI will get it right everywhere, or you can get them in photoshop and do the selection manually.

pjgalbraith• 4 years ago

Conditioning mask strength helps with this. I did a demonstration in this video https://youtu.be/yUfhvPBURSo

emptybits• 4 years ago

A composite without compensation for at least 1) direction of light sources, 2) colour temperature of light sources, and 3) perspective matching will just look "off". Uncanny valleyish. Maybe that's okay or desired for a very limited application. I'm thinking pulp scifi cover artwork, lol.

Humans make these mistakes with naive/low-effort composites. Poorly generated AI images could contain such mistakes also.

OTOH, appropriate training could not only allow AI to avoid the mistake in its own prompt-driven images but also allow it to correct human artist-drawn images. Like local white-balance adjustments or local lens/perspective distortion correction.

krisoft3• 4 years ago

The original, 1990s Myst had 2,500 images, which would cost me more than I expect to make this year. Also, it would take an absurd amount of time.

Myst was computer rendered. If that is the feel the author is going for why don’t they use the same technique? The number of individual images would certainly not be a limit then.

agildehaus• 4 years ago

Indeed. It's also quite a bit easier, faster, and cheaper to make INSANELY better quality images than it was in 1992 when Myst was being developed.

The software and hardware differences between 1992 and today are immense.

What hasn't changed though is the artist. Robyn Miller and Chuck Carter squeezed every ounce out of what was available to them.

This guy could do the same with Stable Diffusion, but will take work it doesn't seem like he wants to do. He wants Stable Diffusion to be his perfect human-quality artist, and it isn't that.

pie_flavor• 4 years ago

My first attempt with SD2 was my favorite prompt in SD1 - 'nebulaic sky above a golden sea, 4k desktop wallpaper'. SD1 routinely made the nebula golden instead of the sea, but what the heck, still makes for a great Zoom background. But SD2 just gave me a golden nebula. No sea. I tried everything I could think of to vary the prompt, and nothing would produce an image with the sea in it.

Then I ran a sanity check, and asked it 'Sea'. I got some interesting abstract patterns, a comic strip about an airplane, and a bird that looked rather diseased. No seas. Maybe one of the abstract patterns kind of looked like the sea does in anime?. Just to double-sanity-check that prompts don't need to be more refined than that, I searched 'forest' and got some lovely photorealistic pine trees in the mist.

SD2 literally doesn't know what the sea is. I just checked and SD2.1 gives you more sea-ish abstract art than SD2.0 did, but still no dice. 'photorealistic sea' gives me two motorcycles and an elephant.

edit: Seems v2.1 understands 'ocean' some of the time (not all!). v2.0 did not understand either one.

Nition• 4 years ago

two motorcycles and an elephant

Maybe SEA = Southeast Asia.

ralfd• 4 years ago

What happens with the prompt "nebulaic sky above a golden ocean, 4k desktop wallpaper'"?

jemmyw• 4 years ago

maybe most images are labelled "ocean"?

TillE• 4 years ago

I'd been using SD1 for fantasy game character portraits, with fairly decent results. SD2 refuses to give me a simple headshot no matter what I try, it's just generating pictures which include most of the body as well.

astrange3• 4 years ago

Try using negative prompting, especially negative textual-inversion prompts.

eightysixfour• 4 years ago

SD2 had the NSFW filter set incorrectly in the image pipeline, it basically eliminates any images with people. Download 2.1, it is much better.

dzink• 4 years ago

Have you tried using ocean instead of sea?

rockzom• 4 years ago

Wonder if it thinks it's an acronym or city. Southeast Asia, Seattle, etc.

astrange3• 4 years ago

v2.0 is missing a lot of data due to a bug. 2.1 restores it.

Besides that, you can search the dataset with https://rom1504.github.io/clip-retrieval/ or create probes with textual-inversion instead of sticking with existing words.

(Oddly, I don't know of any working image-inversion tools. There's one called "CLIP Interrogator" but it doesn't work very well, since it relies on an older image-to-caption model called BLIP that isn't an inverse of SD.)

dcj4• 4 years ago

I'm not.

w4ffl35• 4 years ago

SD works best when you combine it with a paint program.

for example, I generated superman but part of his cape was missing. I brought the image into GIMP and manually painted his cape on (sloppy, with pure red, no attempt at making it nice).

Then I ran the image through "img2img" and the cape looked perfect.

sophrocyne2• 4 years ago

This is one of the the main reasons for InvokeAI building the Unified Canvas

https://youtu.be/hIYBfDtKaus

dorkwood2• 4 years ago

I find this attempt to create something of value without putting in any effort to be frustrating.

Yes, I could pay an artist to make art for the game. But I’m not yet good enough at game marketing for this to make sense.

There's a third option: learn to make the art yourself! Yes, it's time consuming, but so is anything worth doing.

The original, 1990s Myst had 2,500 images, which would cost me more than I expect to make this year.

Start smaller. Or use a simpler, more achievable art style.

oneoff7862• 4 years ago

There's a third option: learn to make the art yourself! Yes, it's time consuming, but so is anything worth doing.

This is terrible logic. If you don’t want to make art, don’t waste time learning to make art.

dorkwood2• 4 years ago

If you don’t want to make art, don’t waste time learning to make art.

Oh, I agree. I wouldn't suggest someone who wasn't interested in art spend time learning to make art. That would be ludicrous.

But that's not what I'm seeing in this article. Most of it chronicles the author's attempt to create a single illustration in a particular style. What are they doing, if not attempting to make art?

wokwokwok• 4 years ago

? Is this a re-post?

I swear this is from like 2 days ago, and it turns up in the HN search from two days ago:

I am frustrated with Stable Diffusion(https://novalis.org/blog/2022-12-05-i-am-frustrated-with-sta...)
43 points|luu|2 days ago|40 comments <---

..but it's been resurrected now?

42 points by luu 5 hours ago | hide | past | favorite | 38 comments <--- ??

I mean, whatever, but the timestamps seem like they've been edited or something, which is very strange.

fragmede• 4 years ago

Second chance pool

https://news.ycombinator.com/item?id=26998309

elil17• 4 years ago

I'd argue the images he generated are good enough to use in an indie game (based on the expectation that good gameplay would make up for the jank).

nineteen999• 4 years ago

Non artist struggles to make game art with an AI image generator. News at 11.

noduerme3• 4 years ago

So ... this is me as a 42 year old artist, a coder in many languages, an art director for some decent sized companies (probably overpaid while my illustrators are definitely underpaid), and Stable Diffusion enthusiast:

HIRE SOME ARTISTS.

[edit] if you feel like a long discussion of the cognitive failure of AI over human artists, bring it on; if you're too cheap to hire artists, I totally get it!!! Make your own art! Otherwise, take your brilliant idea and pay other people to execute the parts you're not capable of executing yourself! Don't expect someone else to hand you a robot/AI replacement for your lack of skill for free ;)

badsectoracula• 4 years ago

He brings that up at the end of the article:

Yes, I could pay an artist to make art for the game. But I’m not yet good enough at game marketing for this to make sense: The original, 1990s Myst had 2,500 images, which would cost me more than I expect to make this year. Also, it would take an absurd amount of time.

noduerme3• 4 years ago

I spent 8 years developing all the art for my own game platform and another 4 years on the next... so .. "also, it would take an absurd amount of time" ? Yeah? Well while you search for shortcuts or hope for a free ride, you're not making the art you need or want. What constitutes an "absurd" amount of time is pretty relative. If you hope to make a sizeable amount off your game, maybe it will take what seems right now to be an absurd amount of effort. It only seems absurd because you haven't connected with the reality of spending your time 24/7 making things. i.e. putting out the effort to earn the acclaim you hope for.

Slightly a side story, so no one complains: In 1994 when I was 14, "Heart of China" came out and VideoSpigot came out and all I wanted for Xmas was a video camera and an interface to record short QuickTime movies in locations around my house, so I could string them together into a dark futuristic CD-ROM game. (in SuperCard, if you're wondering). That would have solved all my problems creating art!

It wouldn't have. Make your game as a sketch with whatever sketching tools you have...but don't expect technology to rescue you from needing other humans to make it good. AND DON'T UNDER-RATE OR UNDERPAY THOSE PEOPLE.

Because trust me, if you're disappointed that AI isn't giving you good enough art, you're no visionary. You haven't even tried people.

pram• 4 years ago

The example is actually 2500 screenshots of a 3D rendered environment. An artist didn’t actually draw every single frame. Kinda misleading imo.

rmetzler• 4 years ago

I would look for an artist to partner with. Build a demo of 1 to 10 levels. Find finance partners.

Danieru• 4 years ago

Shhhh, you're giving away our industry secrets!

No no, everyone hoping to make games should continue to spend years hunting for shortcuts. That one elusive art style which is easy to make and requires no art direction.

jasfi• 4 years ago

I'm tackling this problem with InventAI: https://inventai.xyz. Doing everything I can to launch the MVP this year!

Prompt engineering isn't what it should be.

It will use DALL-E 2 to start with, but Stable Diffusion 2 is the next back-end I'll add.

ChaitanyaSai2• 4 years ago

We've been experimenting with SD and results are quite underwhelming. DALLE2 seems a lot more usable. Any reason why most are sticking with SD? (If that assumption is true)

trifurcate• 4 years ago

Openness for me. Stable diffusion runs on a machine that sits 5 feet away from me and delivers good quality results in under 3 seconds per image. I don't have to shy away from politically (or otherwise) questionable prompts, or in the case of business applications, potentially leaking sensitive data, because I know that they aren't being recorded forever on a remote server somewhere. I can also tinker with the code, which has helped me add a bunch of features that really synergize with my workflow.

As others have noted, getting good results from SD is very much a numbers and parameters game. I've witnessed first hand how some people get frustrated easily and quit tinkering because their initial approaches don't work right away. Prompt and parameter engineering really is a skill that you have to grow, and it makes a world of difference. img2img is also much finickier than text2img.

speedgoose2• 4 years ago

StableDiffusion produces better pictures once you get what you want. Dall.e2 is very good at understanding your prompt but the images are not very detailed when you look closely. They are full of artefacts, and the faces are zombi like unless they fill the whole frame.

But you can use stable diffusion and dall.e together. Or use MidJourney v4 which is simply the best right now.

ChaitanyaSai2• 4 years ago

How does MidJourney v4 do illustrations. The early versions we saw seemed a little too artistic and also had this distinct "looks like midjourney" signature. Any links you can point me to? Thanks!

nitwit005• 4 years ago

Points out the original Myst had 2500 images. Even if an AI tool was so good it only took you 10 minutes per picture, that'd be around 416 hours of work, or 52 eight hour days. Presumably some things will need to be reworked, so probably quite a bit more.

It'd probably be easier to create an ordinary 3D game world. I'm not going to claim 3D art is easy, but you only need to make your Myst island one time, and then you can just move the camera around.

prakhar897• 4 years ago

I'm with the author on this one. I'm working on a Stable Diffusion based word guessing game (https://diffudle.com/). If you look at the prompts and the actual images, they are miles apart. This is after I reject multiple prompts and multiple images from the selected prompt. The worst part is Midjourney is able to create much better images from the same prompt.

At this point, I'm seriously considering renaming the product and just move with another image generator.

thot_experiment• 4 years ago

I dunno man, I put it into img2img and it worked on like the third try. https://imgur.com/a/tbqz59G

    a simple rickety dock at the base of an ocean cliff, approach from the sea, picture from a boat, an illustration by moebius, complexla style, an outdoor scene, simple, expansive, ocean, cliff, stairs, dock, realistic colors
    Negative prompt: isometric, photograph, boat, multiple frames, abstract, complex, borders
    Steps: 20, Sampler: DPM++ 2M Karras, CFG scale: 7, Seed: 1136556607, Size: 1024x1024, Model hash: 5e107da3, Denoising strength: 0.51, Eta: 0.4, Mask blur: 4

jerpint• 4 years ago

Wow impressive! I need to learn more about negative prompting

kecupochren3• 4 years ago

Try Midjourney instead. It's miles ahead. Very little is needed to get amazing results. You just set the theme and let it run wild.

anentropic2• 4 years ago

Are there different versions of midjourney or something? I tried it last week and the results were equally underwhelming

kecupochren3• 4 years ago

Yes, you need to use --v 4 parameter to use the latest version

_ph_• 4 years ago

What I always find so disappointing is, that you might get an almost perfect image out of it, but then there is something decisive very wrong. Like a fifth leg on an anmial oder persons with 3 arms. Or the face completely missing in an otherwise good rendering.

So while the results often are astonishing - I am still faszinated that it is possible to create realistic images just from a short text input - this shows that there is no true intelligence behind this. It will be interesting when this gap can be breached.

Till then, I can hope that there will be good tooling around SD, which make image creation easier by offering a good UI to make iterations easier and allow the user to guide the creation better.

avereveard• 4 years ago

eh, it's all anectdotes unless he shows us his prompts

boredhedgehog• 4 years ago

I don't know much about SD, but the author might be confused about the technical terms. He wants a pier but keeps asking for merely a dock.

Fricken3• 4 years ago

Here is my first attempt at "wooden pier beneath sea cliffs by Moebius --v 4", in Midjourney:

https://imgur.com/KnNCFJQ

cloogshicer• 4 years ago

When I see things like this, I always have to think of this Scott Aaronson quote:

"If P=NP, then the world would be a profoundly different place than we usually assume it to be. There would be no special value in “creative leaps,” no fundamental gap between solving a problem and recognizing the solution once it’s found. Everyone who could appreciate a symphony would be Mozart; everyone who could follow a step-by-step argument would be Gauss; everyone who could recognize a good investment strategy would be Warren Buffett."

Maybe the difference between recognizing art and making it yourself is not as big as we assumed it to be.

smcl• 4 years ago

Don’t you know how solid objects work?

sophrocyne2• 4 years ago

There is a significant learning curve to using SD - Not just in developing the craft of prompting, but recognizing that fine-tuned model choice and tooling choices play a significant role in output quality, and the ability to control the generation process.

The more you abstract control from end users and “do it for them”, the easier generations can be made, at the loss of control. Midjourney is an excellent example of this.

If you want fine-tuned control, SD has an ecosystem that is rapidly accelerating in capabilities.

I’m part of a team building open source toolkit on top of SD, and the power users of our tool have shown themselves to be exceptional in their ability to pull high quality works out of the aether.

woolion• 4 years ago

I made a similar post [0], hoping to 'replace myself', but being really underwhelmed with the results. And this is after a few full days of prompt hacking, Dreambooth training on different base models, etc. It's clearly not at a level where you can easily replace even low-tier artists in even semi-pro environments.

It's not the case however for things like one-off illustrations (books, music albums) where it's easy to see that indies have totally embraced it over commissions already. It doesn't require general consistency, nor to be very specific, nor to have a very unique design.

http://woolion.art/2022/11/16/SDDB.html

I am frustrated with Stable Diffusion

Related Stories