Show HN: AI-Generated Illustrations of Plants

https://www.easyzoom.com/imageaccess/2128f27845ed4921b3143004dc550c3e

by Yenrabbit • 5 years ago

137 61 5 years ago

Yenrabbit18• 5 years ago

Hi HN, I've been playing around with AI art (who hasn't) and wanted to share the results of last nights run: 300 auto-generated plants in a big collage that you can zoom into and explore. My end-goal with this is to work on some sort of speculative biology project. GANs like the one I used here are great for generating nice-looking images in bulk, which could then be curated and refined into some sort of consistent work. Happy to answer any questions on how this was generated or on AI art in general :)

fumblebee4• 5 years ago

This is cool! Thanks for sharing. I've played around with some random Colab notebooks that have surfaced on HN but the results have been underwhelming compared with some of the polished AI Art I've seen in the wild. Some questions that popped into my head:

What's your setup (Cloud/Colab/Custom hardware)? Did you borrow the code in its entirety or is there a secret source? How long did it take you to fiddle around with the hyper-parameters until you were happy with the results? How many iterations did you settle on before stopping? Thanks!

Yenrabbit18• 5 years ago

It can be hit-and-miss. With a bit of trial and error you start to see which prompts work well to generate pleasing images. Some people have done excellent work exploring ways to change the look, like https://imgur.com/a/SnSIQRu

As for hardware, a lot of my experimenting is done on Google colab. For this plant generation stuff I rented a GPU via vast.ai for ~$0.20 an hour and set it running overnight.

thanatos5192• 5 years ago

The '8k 3D' examples there are crazy red/blue analyphs. Total brainwarp.

fumblebee4• 5 years ago

That link is incredibly extensive, great resource!

I find the "tattoo" effect to be extremely disturbing.

Yenrabbit18• 5 years ago

re: secret sauce, I tend to be in the 'sharing is caring' camp. The code for this was based on the popular notebook by @RiversHaveWings (VQGAN+CLIP) although I've edited it back and forth a few times. I usually run for a few hundred iterations (eg 250). EDIT: here's a Google Colab that replicates my plant generation: https://colab.research.google.com/drive/1b1UfblpdhPJ7f1WRjfC...

fumblebee4• 5 years ago

Thanks for taking the time to reply to my Qs; all useful insights and is much appreciated.

Mostly I'm excited about the direction and innovation velocity that AI art is going, but partly I'm anxious about what the eventual implications will be for human artists.

jcims3• 5 years ago

I'd like to humbly suggest that you give her CLIP guided diffusion model a try. It's blowing my mind.

https://colab.research.google.com/drive/12a_Wrfi2_gwwAuN3VvM...

Some recent output (prompts are summarized)

max max alien craft in the desert - https://imgur.com/a/1R5UZQb

worshippers gather at mad max alien craft in the desert - https://imgur.com/a/acdbLVu

vintage 70s alien spacecraft in the meadow - https://imgur.com/a/kwAVWG9

(note these are all upscaled with waifu2x)

My guess is that you've solved this already but I run it in a loop by replacing do_loop() at the bottom of the last frame with the following:

    import random
    prompt='crystalline alien spacecraft on a flower'
    modifiers=['',' macro photography', ' with hyperrealistic 35mm depth of field']
    original_prompt = prompt
    while True:
      for modifier in modifiers:
        prompt = original_prompt + modifier
        seed = random.randrange(1000)
        tqdm.write(f'Prompt {prompt}, Seed {seed}')
        do_run()

malka• 5 years ago

Packaging is meh, but you can use my repo if you want to generate your images using the CLI, a discord bot, or an IRC bot with your own hardware: https://github.com/luc-leonard/clip_generators. It includes vqgan+clip and guided-diffusion models

ak391• 5 years ago

there is also a easy to use web demo here on huggingface with some presets https://huggingface.co/spaces/akhaliq/VQGAN_CLIP

Yenrabbit18• 5 years ago

This is what I send to non-technical folks who want to dabble - thank you for making that available :)

graderjs• 5 years ago

Looks very alien! Great!

lyaa• 5 years ago

I love these so much! How did you direct the GAN to generate them?

Yenrabbit18• 5 years ago

I made a prompt generator that creates descriptions like 'A biological illustration of an alien plant near sand, watercolor'. A model called CLIP gives a way to compare an image with a text prompt and then I can optimise the image to match the prompt

datameta• 5 years ago

You've been able to get some interesting output! I've been playing with the following notebook: https://colab.research.google.com/drive/1oA1fZP7N1uPBxwbGIvO... but my results are not quite so well put together as yours. What are some tweaks you've made to the model? I'd like to better understand what is going on under the hood.

baxtr• 5 years ago

Awesome. Could you also do fruits?

PS: thanks for not calling it „this plant does not exist“

Yenrabbit18• 5 years ago

I'd be lying if I said I hadn't considered setting up a website at thisplantdoesnotexist.com haha luckily I changed my mind. I could try some fruits - let me set a smaller run going and see that it comes up with.

Yenrabbit18• 5 years ago

I tried generating some fruit - you can see some outputs in the notebook I just shared (https://colab.research.google.com/drive/1b1UfblpdhPJ7f1WRjfC...). Not quite as pleasing as the plants but I'm sure with a bit of fiddling with the prompt we could get a few good ones.

skinkestek• 5 years ago

To me this is probably the most interesting / delightful.

The one with persons is useful, this one is something I'd like to order as a print or maybe even as a wallpaper for my company headquarters (haha, it is currently <<10m2) sometime in the future :-)

Yenrabbit18• 5 years ago

I'm happy to share the original JPEG if you want to print this out :) I have it in A2 on my wall although at that scale the individual pictures are a little small.

ectopod2• 5 years ago

Please do, that would be brilliant.

thanatos5192• 5 years ago

I love how the plants seem to flow into the landscape in many of these!

totetsu2• 5 years ago

Am I a curmudgeon for wanting to call most of these ai art projects ai illustration projects?

Yenrabbit18• 5 years ago

No, I think that's fair in this case. Most of my stuff involves a lot more post-processing, tweaking, editing etc to convey something specific, in which case I claim 'art' more liberally :)

totetsu2• 5 years ago

https://www.hicetnunc.xyz/ nice stuff

eqbridges• 5 years ago

Reminds me of the work called “Herbarium” by Spanish artist Joan Fontcuberta: https://www.juanmagonzalez.com/fontcuberta/herbarium.html

namero999• 5 years ago

And to me, it reminds of the Codex Seraphinianus

fsiefken• 5 years ago

and to me the voynich manuscript, perhaps this AI plant generation could provide an additional clue to the meaning of the fantasy plants in the manuscript!

Yenrabbit18• 5 years ago

This was one of the things that got me hooked on the idea of an encyclopedia for an imagined world! Such a fascinating and fun work.

illegalmemory• 5 years ago

Thanks for sharing, this is cool! But in most of such projects I have seen is lack of continuity in generated content. From a bird eye view things look natural but as soon as you observe the content, a little closer, things start to fall apart.

Yenrabbit18• 5 years ago

Agreed. When I'm using this as part of a larger workflow I'll sometimes use the GAN to generate a starting point and then manually edit things to remove weird inconsistencies. The output is also quite low-res so it needs tweaking. Of course both problems can be avoided by making the individual images so small that the weirdness is hidden - as I did for this project :)

smusamashah• 5 years ago

They look like drawings in Codex Seraphinianus. See these images https://images.google.com/search?q=codex+seraphinianus+plant...

6174• 5 years ago

Cool , thx for sharing, if these image can be generated by AI, that will be pretty interesting

baltimore• 5 years ago

A lot of them seem to be growing out of books :-)

Yenrabbit18• 5 years ago

My fault for including 'nature journal' in some prompts!

ramigb• 5 years ago

TIL AI thinks plants and reproductive organs look a like

thaumasiotes• 5 years ago

Wait till you hear about orchids.

psfourplusone• 5 years ago

Cool, reminds me of Codex Seraphinianus

valyagolev• 5 years ago

check out also Algorithmic Botany, that tries to use algorithmic creation for research http://algorithmicbotany.org/papers/

IncRnd• 5 years ago

Now we know the origins of the Voynich Manuscript.

ryandvm• 5 years ago

I'm curious what the state of AI generated music is. I heard a quote once that "music is math for people that don't like numbers" which struck me as particularly insightful. All the things that people appreciate about the instrumental aspect of music are mathematical in nature (rhythm, chords, progressions, etc.), at least at a superficial level.

Apart from vocals and lyrics (yet), it seems that AI ought to be particularly well suited for creating artificial music that actually sounds decent. Sure it's not going to be pushing any boundaries, but I'd be surprised if it wasn't able to churn out some catchy melodies.

DiggyJohnson• 5 years ago

Really? My initial reaction is to disagree, though I do think I understand your point.

While the medium is music is purely physical, and physical things can be abstracted to mathematics, what do you think about the generation/composition of music. Most music isn't really composed of instruments playing single notes; it's composed of things like phrases phrases and expressions. The math is just an obtuse way of representing the phrases, emotions, and expressions.

What do you think of this perspective?

Yenrabbit18• 5 years ago

Some sounds OK, most sounds like a series of plausible notes that nevertheless don't actually go anywhere. Most generative models tend to do OK at predicting the immediate next note, say, but they can't keep a macro-level structure unless you force that as a constraint. For me what has worked best is generating music and then re-interpreting that with some manual post-processing to get it into a less jarring form. A fairly recent project that is close to the state of the art: https://openai.com/blog/jukebox/ One of my own experiments: https://datasciencecastnet.home.blog/2021/05/13/whistlegen-g...

LargoLasskhyfv• 5 years ago

The result of that is what is played up and down the charts, learned from what sells 'records', and embedded into almost anything new to be more and more refined to the taste of the majority/masses. Also the stuff playing in supermarkets to make you buy more.

popotamonga• 5 years ago

Game art illustrators will be the first to get replaced with AI. At least on non premium titles.

aetherspawn• 5 years ago

These would make a cool postage stamp set.

sampsonferd• 5 years ago

Looks so cool! Love it

laibasidd• 5 years ago

Loved it!! It's pretty amazing :)

WheelsAtLarge• 5 years ago

Can you give us a summary/write up on your process? It's very impressive.

Show HN: AI-Generated Illustrations of Plants

Related Stories