Show HN: This Food Does Not Exist

https://nyx-ai.github.io/stylegan2-flax-tpu/

by MasterScrat • 4 years ago

248 172 4 years ago

nyx-ai.github.io

gus_massa• 4 years ago

Are you using the same model for cookies and cheesecakes? Do you get sometimes a cookiecake?

We currently train each model independently, ie we first gather a cookie dataset, train a cookie model then restart from scratch for the next one.

That's actually something we're investigating: can we train a single class-conditional model for multiple types of food? Or, can we finetune cheesecakes from cookies?

TuringNYC• 4 years ago

> ie we first gather a cookie dataset,

Is there a chance your dataset provider makes a claim that they have derived data rights over your model generated images? Would you have sufficient confidence, say, to sell your images on a stock image site?

dylan6043• 4 years ago

Well, now I want a cookiecake.

ch4s33• 4 years ago

The dream of the 90s is alive in StyleGAN!

MasterScrat3• 4 years ago

We have trained four StyleGAN2 image generation models and are releasing checkpoints and training code. We are exploring how to improve/scale up StyleGAN training, particularly when leveraging TPUs.

While everyone is excited about DALL·E/diffusion models, training those is currently out of reach for most practitioners. Craiyon (formerly DALL·E mega) has been training for months on a huge TPU 256 machine. In comparison our models were each trained in less than 10h on a machine 32x smaller. StyleGAN models also still offer unrivaled photorealism when trained on narrow domains (eg thispersondoesnotexist.com), even though diffusion models are catching up due to massive cash investments in that direction.

goldemerald• 4 years ago

I don't suppose you have a way of converting these models into a pytorch usable version, do you?

forgingahead• 4 years ago

Nice work - can you share how large your training datasets are in terms of number of images? And did you train your models from scratch or fine-tune them from an existing model?

andrewmcwatters• 4 years ago

Darn! I was hoping for other-worldly foods that don't actually exist being generated from real food attributes. I suppose I should have known better.

spacemanmatt• 4 years ago

Somewhere in that data set is found an Eigencookie. I want the recipe.

WalterSear• 4 years ago

No hot dogs?

Nice work.

dylan6043• 4 years ago

When will we see this as a contestant on Is It Cake?

croes2• 4 years ago

You are killing Instagram influencers

Spivak• 4 years ago

You mean supplying. Imagine running a food IG that didn’t even need to make the food.

ch4s33• 4 years ago

Hold my “beer”.

croes2• 4 years ago

Imagine being one of millions, your food IG will look fake even if the photos are real

scifibestfi• 4 years ago

Seriously, won't this combined with GPT-3 flood the influencer market?

bergenty2• 4 years ago

Yes. Images will lose all authenticity.

minimaxir• 4 years ago

If this doesn't, DALL-E 2 will: https://twitter.com/minimaxir/status/1549761827969544192

raphar• 4 years ago

One twit there, made me happy: This is going to break pinterest

forgotusername6• 4 years ago

I wonder how close the nearest match from the training data is. Was there a cheesecake that looked almost like these generated images?

layer8• 4 years ago

Maybe the ML model effectively implements a lossy image database with minor randomization. :)

fxtentacle6• 4 years ago

Since GANs are effectively one class of denoising auto-encoders, your summary is spot-on. This type of ML model learns to effectively compress and decompress natural images by representing it as a hierarchy of convolutional features = shape templates.

mejutoco• 4 years ago

100% It would be interesting to get an input as image and get the closest prompt that generates that image. Later we could run the prompt to get the image back.

That could effectively be a lossy compression of the image, exchanging space (the image) with time (cpu).

nh23423fefe• 4 years ago

This feels like when I made a drawing in elementary school, and someone asks if I traced it. It just feels like looking for a way to downplay what was made by making an appeal to "creativity" or "originality".

But the tide never goes out on AI and computing. The capabilities will only grow more and more impressive and unassailable.

When the chatbot is completely convincing is someone going to ask, "I wonder close the responses are to training text" even though no one even blinks when fathers and sons act alike. No one demands children invent new languages to prove they aren't just "randomizing samplers"

yellowapple3• 4 years ago

No one demands children invent new languages to prove they aren't just "randomizing samplers"

I sure as hell do. No son of mine will be comprehensible to other humans until he's at least two years old.

mikkergp3• 4 years ago

Is this just a search engine to find relevant content and remix it a bit, or can you actually create new content. These two things don't solve the same problem, and you may run into lots of copyright problems.

waynesonfire2• 4 years ago

exactly.. where is the chocolate chip cheesecake?

waynesonfire2• 4 years ago

and what was the licensing for the training data that you used?

ComputerCat• 4 years ago

Everything looks delish!

golergka• 4 years ago

And here I was, hoping for new, never seen before dishes.

Sebbecking• 4 years ago

How big was your training dataset?

n4bz0r• 4 years ago

The food looks great! I suppose these models could use some extra training with dishes, though. The plates and glasses look wobbly, which is an instant giveaway. Otherwise, I can see this being used by food posters! Maybe not as a primary source, but as a "filler" — for sure.

hammycheesy• 4 years ago

I tried to use the linked Colab notebook to generate my own, and it appears to have been successful, but I don't see any way to view the generated images via the notebook interface. I'm not familiar with the notebook tool - have I missed something?

sireat• 4 years ago

If the result is standard numpy 3d matrix then something like Pillow should be able to display the images.

Something like

  from matplotlib import pyplot as plt

  plt.imshow(matrix)

  plt.show()

lancebeet• 4 years ago

I ran it locally and it generated images as PNGs in the "generated_images" directory (named 0.png, 42.png etc. after the seeds provided to the script). If it works and does the same in the notebook, you should be able to click the folder icon in the menu on the left to open the file browser, expand the "generated_images" directory, then click the ellipses next to each file to select "Download".

munificent• 4 years ago

I love cheesecake with strawraspcherries on top.

derbOac• 4 years ago

The Cake is a Lie meme was never so relevant.

mike_hock• 4 years ago

And the Science gets done and you make a neat gun.

beej71• 4 years ago

We're looking at the complete collapse of the stock photography market.

echelon4• 4 years ago

It's way more than that.

Anyone can be an artist, musician, photographer, writer.

It's going to result in more content being created, which will change the economies of content. Rate, scale, and volume of production will increase by orders of magnitude.

Disney thinks IP is a war chest. That's an old way of thinking.

Star Wars won't be special to the new kids growing up that can generate "Space Samurai" and "Galaxy Brouhaha" in an afternoon.

We're going to hit a Cambrian explosion of content.

smaudet6• 4 years ago

"It's going to result in more content being created"

Is it, though? This model took over a month, on extremely fit hardware, to even create.

Lets say for a second, in some hypothetical future, that anyone can access/use/update these models (by anyone, I mean someone with both low amount of resources as well as little to no programming skill), why are they creating content?

"Rate, scale, and volume of production will increase by orders of magnitude."

If by production you mean "paid creation", I'm not so sure about that. In this world where everyone creates content from thin air 1) there is little to no monetary value to the content anymore (as monetary value inversely correlates with scarcity) 2) So there is less incentive to create anything, because there is no monetary value to doing so.

In fact, by definition we can pretty much prove that not much of anything will happen in this regard, because content is already limited by budget - the budget has not gone up, and the return has only gotten worse (in this hypothetical scenario).

What I think is more likely to happen - a few, "blessed" individuals will have out-sized content creation capabilities, without much need to innovate. The rest of us will have almost no incentive to create anything as a result.

Disney will use these systems, and they will use them to churn out more garbage, faster, on average, most kids will not be generating any movies in an afternoon.

nixpulvis• 4 years ago

Friggles and bop, produce nothing.

jeffreygoesto• 4 years ago

It's just that they don't "create" anything and pressing some buttons to get more and more of the same, biased crap quickly gets boring.

Victerius• 4 years ago

I have some extremely detailed imaginary images and clips in my head that I just don't want to devote the thousands of hours it would require to become proficient enough in drawing and visual effects to create them.

gffrd5• 4 years ago

"Smiling happy family holding cell phones above their heads standing in a field of grass wearing all white"

dalmo32• 4 years ago

That's actually a great example. Just think of the aggregated human-hours wasted to bring those people together, create that setting, photograph, edit, publish... All for a meaningless flyer or landing page.

Cockbrand• 4 years ago

Yields subtly nightmarish results:

[1] https://i.imgur.com/YqNkaGj.png

[2] https://i.imgur.com/EQL7pqw.png

gffrd5• 4 years ago

"Boardroom full of attractive business people gathered around a laptop with one of them pointing at the screen, all wearing suits, whiteboard in the background"

robotresearcher• 4 years ago

"Woman laughing alone with salad"

corrral8• 4 years ago

Thinking bigger: I'm pretty sure the combo of a relatively free global Internet, liberal democracy on large (much bigger than city-state) scales, and cheap, customized, on-demand generation of totally fake text + photo + video propaganda based on simple prompts, cannot all co-exist. At least one of these isn't going to survive alongside the others. If we just let things keep going the way they are, I expect "liberal democracy on large scales" is the one we'll lose—and whatever follows probably won't let the fairly-free, global Internet keep existing, either, so we'll lose that too.

blueflow• 4 years ago

Whats your reasoning on this? Because i dont see why liberal democracy would cease to exist... life would go on if we all know that all pictures can be fabricated. I think this is already the case without AI.

konschubert• 4 years ago

You can put letters in any order you want and make them say any damn lie.

This was not an impediment to liberal democracy.

I am as concerned as the next guy but throwing the towel already seems a bit premature?

whatshisface• 4 years ago

I have heard this before, but I have several reasons to think it is not going to be a problem above what already happens:

1. AI lets you generate an enormous number of lies, but what is really dangerous is one well-placed lie within a trusted stream of many truths. CNN will retain a power to mislead far in excess of Twitter bots.

2. Democracy averages over everyone's confusion, which means lies are only dangerous when large numbers of people believe them at the same time. Hordes of bots generating spurious lies won't move a democracy in any specific direction, but again, mass media will retain its power to mislead everyone at once in the same, effectual, direction.

3. People have never respected the veracity of random tweets. In the same way that trust in mainstream media outlets is reaching record lows due to their consistently biased reporting (they might not all have the same bias, but I can't think of any I'd consider free of bias), everyone will learn to adjust their incredulity to match the true quality of random tweets.

4. Companies like Twitter and Google are known to be shaping their results and algorithms according to their own "political views" (broadly construed) so at worst this would represent a partial shift of power from the old masters to new masters quite like them (social media companies). In many ways trimming the front page to reflect editorial opinion is echoed in the way Twitter trims its feeds to reflect their own editorial opinions.

All taken together, it seems like the media is afraid that equally large companies with similar business models (content, attention, advertising) might end up eclipsing them. The same old model where the TV station is afraid to upset its advertisers, thereby giving a voice to business interests, is well-recorded in the recent history of YouTube. Not so much will change, although seeing it in its old and newer forms might shed light on how it works.

hwers2• 4 years ago

Lol imagine if someone said this as a prediction when photoshop was invented.

nomel3• 4 years ago

I disagree completely. We've had this ability to create photorealistic fiction for decades now. You can go see how impressive the capability is by paying $12 at your local movie theater. The only thing that's changed is the ease and cost of doing it, but that has also been dropping for decades. Having people think that it's not possible right now is what is dangerous, because it's absolutely possible right now, no AI needed.

Even if this is released quickly, the internet would be flooded with crazy images, and people would quickly learn to have a critical eye to the media that they see, which should already be the case, because photorealistic fiction is already possible, as everyone experiences with every blockbuster movie. Only a slight shift in perspective is required to realize that every image and video you see aren't any different than frames/clips of Thor: Love And Thunder.

fxtentacle6• 4 years ago

Partially, yes. I certainly predict that DALLE-like models will ruin the prices for some stock photos.

But on the other hand, Adobe is pushing their CAI hard:

https://helpx.adobe.com/photoshop/using/content-credentials....

And the core benefit of "authentic" content is that it can't be generated by an AI. Only humans can own copyright.

imachine1980_• 4 years ago

dall-e is start giving full commercial license , i think in the end both will converge you will prom something the AI will make 100 prototypes and you will improve the one that you like whit help the AI. the line will blur, is not against the machine but whit the machine. the problem maybe is that will probably need less people to do the same works. is "fun" maybe is better be construction worker than artist, becouse the second will get obsolet for most case.

Kye2• 4 years ago

Shrinking microstock rates already killed it.

gnicholas• 4 years ago

This will also democratize the market for comics. It used to be you needed to be able to draw to be a comic. Now you can just have ideas, and use This Comic Does Not Exist (which does not yet exist) to generate the imagery.

kerblang• 4 years ago

To what extent are we ripping off the photographers? Weren't the models trained on their hard work?

Have we reached a point where we've bounded art within the data models are trained on?

Have we imposed a limit on ideas as a realm of "what came before" and implicitly decided that any "after" is a pointless exercise without knowing whether that's even true?

nathanaldensr• 4 years ago

Excellent questions, and I was thinking the same thing. In my opinion, AI-generated art or images are not as impressive as they might seem at first purely because there is no real imagination involved. It's an art simulacrum.

A more accurate title would be "This picture of food does not exist."

SamBam2• 4 years ago

To what extent are we ripping off the photographers?

Photographers in general, certainly, but no individual photographer enough that it could ever matter, legally or ethically.

implicitly decided that any "after" is a pointless exercise...

Seems a little pessimistic. I believe that we will still be able to recognize and appreciate new art.

hammock• 4 years ago

Can these things be used for commercial purposes? Which ones?

jowday• 4 years ago

For a generative model to hit the quality threshold you see on this site, it’s going to need so much training data that you can presume there’s already a ton of stock photos that match that description that it scraped from the internet.

djmips• 4 years ago

Stock market crash!?

labrador• 4 years ago

I'll never trust stock photos again

notamy• 4 years ago

I was really hoping that this would be never-before-seen, AI-generated recipes or something similar ):

sovok• 4 years ago

That would be the OpenAI Recipe creator (eat at your own risk) https://beta.openai.com/examples/default-recipe-generator

JadoJodo3• 4 years ago

OP: Forgive me if this is out of place. Also, please know that my question is genuine, not at all a reflection on the author/their project, and most certainly born out of my own ignorance:

Why are these kinds of things impressive?

I think part of my issue is that I don't really "get" these ML projects ("This X does not exist" or perhaps ML in general).

My understanding is that, in layman's terms, workers are shown many, many examples of X and then are asked to "draw"/create X, which they then do. The corollary I can think of is if I were to draw over and over for a billion, billion years and each time a drawing "failed" to capture the essence of a prompt (as deemed by some outside entity), both my drawing, and my memory of it was erased. At the end of that time, my skill in drawing X would be amazing.

_If_ that understanding is correct, it would seem unimpressive? It's not as though I can pass a prompt of "cookie" to an untrained generator and, it pops out a drawing of one. And likewise, any cookie "drawing" generated by a trained model is simply an amalgam of every example cookie.

What am I missing?

bee_rider2• 4 years ago

For the longest time it was assumed that creativity was an almost magically human trait. The fact that somebody can, with a straight face, say "I don't get why it is impressive, I could draw these images too" is actually indicative of the wild change that has occurred over these last couple years.

I guess it is true that more than a couple demos like this have been shown, so some of the awe might have worn off, but it is still pretty shocking to lots of us that you can describe the general idea of something to a computer and it can figure out and produce "what you mean," fuzzy as that is.

JadoJodo3• 4 years ago

The fact that somebody can, with a straight face, say ...

To be clear, I'm not trying to devalue this at all; In fact, as I noted above, I am certain I'm missing something and that was what my comment was aimed at. In any case, thank you for taking the time to reply (seriously).

mikkergp3• 4 years ago

I will say that the images included have not show to be particularly creative, unless I missed a wider galaxy of non-existent food items. It's not entirely convincing that the generated images aren't just glued together pieces of other images with some fading between them.

Tomis023• 4 years ago

For the longest time it was assumed that creativity was an almost magically human trait. The fact that somebody can, with a straight face, say "I don't get why it is impressive, I could draw these images too" is actually indicative of the wild change that has occurred over these last couple years.

It's not creativity though. It's a program that arranges pixels in a way that is statistically similar to some training data set. It doesn't "draw" anything, it doesn't "figure out" anything. There is no thought or idea behind it.

The output is mildly interesting but there is no creative act at work, and there's certainly no revolution in the artistic world.

For the longest time it was assumed that creativity was an almost magically human trait.

Maybe by some people. This is a human-centric perspective, a form of speciesm if I can call it that. Various primates have shown creativity, and various animals have shown the ability to solve problems creatively. Heck, even my cat figured out how to open doors by pulling down on the handle. Humans are likely not more creative than other animals with similar brain size, it's just that there's other factors at play that make it seem like that (such as opposable thumbs and the passing down of knowledge between generations using speech).

woojoo666• 4 years ago

Regardless of what people think as impressive, or creative, or sentient, or intelligent, there is one thing that cannot be disputed: stock photos have value in the current economy. There are people being paid to produce these stock photos. And it now looks like AI can do it for cheaper. That's the most important thing to realize imo, is that the value of human production is getting lower and lower relative to AI.

NoPicklez• 4 years ago

And think about on demand ML, where a stock photo on a website might change based on characteristics it knows of you.

Let's say a website knows you love chocolate cake, well the website is now going to show you a chocolate cake instead and has generated a unique image of that for you.

Smoosh• 4 years ago

I think the real value is that it can combine items. I can easily find a stock image of a cake. I can probably find a stock image of a circular saw. But a decent stock image of a cake in the shape of a circular saw maybe doesn't exist anywhere. But this could generate several for me to choose from.

And then there's the "in the style of Salvador Dali" aspect.

fancymcpoopoo2• 4 years ago

"AI" can do this for cheaper because it the model is built based on databases of stolen material. did they get any permission from all the people involved in creating the reference photos?

staticassertion• 4 years ago

It's a computer. That's the impressive bit.

XCSme• 4 years ago

The process is not important, the idea is that a computer is able to generate new, never seen before, content that a human can understand.

This allows for incredible things, like having a world-class artist create a song drawing based on your prompt (i.e. DALL-E) in one second.

gffrd5• 4 years ago

I like the thought that, years from now, we're all drinking eating weirdly-presented food / drinking weird cocktails because AI synthesized the images of drinks around the web and decided `cocktails always include fruit` and `all food must be piled high on plate`

wyldfire• 4 years ago

Are there any analysis techniques that can easily distinguish between these and real photographs? Do simple things like edge detections or histograms reveal any anomalies?

daveguy• 4 years ago

Neural networks can be trained to identify the difference, but I don't know how specific that is to the generating model. In fact, the GAN technique, at a high level is two networks -- one trying to distinguish the difference and one trying to create images that cannot be distinguished. That is the "adversarial" aspect.

It is an interesting question that there may be some simple pre-processing techniques (edge detection, Fourier transform, etc) that more easily distinguish the image as a fake. Something like a shortcut from training a network to make the distinction.

fxtentacle6• 4 years ago

I'm honestly surprised that they trained a StyleGAN. Recently, the Imagen architecture has been show to be both easier in structure, easier to train, and even faster to produce good results. Combined with the "Elucidating" paper by NVIDIA's Tero Karras you can train a 256px Imagen* to tolerable quality within an hour on a RTX 3090.

Here's a PyTorch implementation by the LAION people:

https://github.com/lucidrains/imagen-pytorch

And here's 2 images I sampled after training it for some hours, like 2 hours base model + 4 hours upscaler:

https://imgur.com/a/46EZsJo

* = Only the unconditional Imagen variant, meaning what they show off here. The variant with a T5 text embedding takes longer to train.

gwern• 4 years ago

Or, since they are comparing to Craiyon, why not just finetune Craiyon itself? Craiyon already exists, just take it off the shelf, you don't need to retrain it from scratch, so the cost to train it from scratch on everything (which is indeed quite large) is not relevant to someone who just wants to generate great food photos.

MasterScrat3• 4 years ago

We haven't experimented much with Imagen, but our initial conclusions were that:

- It's hard to train to a photorealistic quality (we'd be happy to be proven wrong!)

- There is no strong pretrained model available yet

Checking the LAION Discord, the situation doesn't seem to have considerably evolved.

xg15• 4 years ago

At least with DALL-E you can be sure the food has a name. For a moment I was worried this would produce vaguely food-like images where on closer look you realise you have no idea what you're looking at - like a lot of other "this X does not exist" projects seem to do.

Also a bit of cultural bias in the training is shown I think. The "pile of cookies" prompt seems to mostly generate American cookies, while e.g. a German user might be disappointed they didn't get this: https://groceryeshop.us/image/cache/data/new_image_2019/ABSB... :)

fxtentacle6• 4 years ago

I thought DALL-E uses a sentence-piece encoder for the text that goes into CLIP, which would suggest that you can recombine the syllables from existing words and it'll "understand" that.

So both "banana chocolate cookies" and "banacoochoconakieslade" should work.

dalmo32• 4 years ago

I don't think a German user writing "pile of cookies", in English, would be disappointed with "English" results. Is that any different than what you get on, say, Google?

Try prompting craiyon for "Ein Stapel Kekse"* :)

* Google-translated

uptown• 4 years ago

Like some alien sushi?

https://twitter.com/khirasaki/status/1543111054460272640

YeGoblynQueenne3• 4 years ago

Nice, very alien, somewhat Giger, but not very sushi.

herpderperator• 4 years ago

Is there a way to trigger a fresh image on demand? That's kind of what I expect when I see a does-not-exist site.

CSMastermind• 4 years ago

There's a link on the page:

https://colab.research.google.com/github/nyx-ai/stylegan2-fl...

georgeburdell• 4 years ago

This is the most disturbing “does not exist” yet. A food blog could write itself

hbn• 4 years ago

They already pretty much are. Top recipe hits on Google seem to always be from like "Southern Mama Cooking Tips" or something generic like that, and you have to scroll past 8 paragraphs of context for why this person is writing a recipe and why they like it so much, totally not to hit all the SEO sweet spots, and the full life story of this "Southern Mama" that's totally not a guy in India or a robot scraping together blurbs of text from other website.

WordAWarning• 4 years ago

It's not entirely SEO, that's part of it, but it's also copyright.

You can't copyright a recipe. It's information that can be freely shared. Anybody can steal it and set up their own site. You can copyright a recipe that comes with a life story. Copyright laws are weird and confusing, but it's really difficult to come up with a better solution.

tmountain• 4 years ago

Aggregate "does not exist" website for anyone who's interested.

https://thisxdoesnotexist.com/

jdthedisciple• 4 years ago

Looks impressive but I can't escape the notion that surely some of the generated images will be very close to the some of the training images?

How am I to assess how original the generated results really are?

danuker• 4 years ago

Image search, I guess. No results, it's original enough.

hwers2• 4 years ago

Yeah stylegan is rarely this good at super diverse data like this

rkagerer• 4 years ago

My partner is very impressionable when she see's food in a TV show. Immediately has a craving for it. This thing is like, limitless porn for her gluttony.

twic• 4 years ago

This computer has pretty poor taste in cocktails.

Animats• 4 years ago

Coming soon to the restaurant site generator of some large delivery service.

("Picture is only for illustration purposes")

abidlabs• 4 years ago

You can try out the model with this interactive Gradio demo: https://huggingface.co/spaces/nyx-ai/stylegan2-flax-tpu

immmmmm• 4 years ago

should call that DeepCakes

Show HN: This Food Does Not Exist

Related Stories