cannoneyed

a) couldn’t decide where to draw the edges b) I live in NJ ;)

Oh wow this is great, I threw a few of these tricks at it and never really felt like I nailed it... Might have to revisit after playing around with this more

I just couldn't justify spending dozens more hours for Staten Island...

This is so good, just ordered one!

I'd say that deciding where a transient should go" is creative, manually aligning 15 other tracks over and over again is not (not to mention having to do it in both the DAW and melodyne)...

I agree that "push button get image" AI generation is at best a bit cheap, at worst deeply boring. Art is resistance in a medium - but at what point is that resistance just masochism?

George Perec took this idea to the max when he wrote an entire novel without the letter "E" - in French! And then someone had the audacity to translate it to English (e excluded)! Would I ever want to do that? Hell no, but I'm very glad to live in a world where someone else is crazy enough to.

I've spent my 10,000 hours making "real" art and don't really feel the need to justify myself - but to all of the young people out there who are afraid to play with something new because some grumps on hacker news might get upset:

It doesn't matter what you make or how you make it. What matters is why you make it and what you have to say.

Yup, I'm a big fan. They'd be the first to admit that making true pixel art is very hard, and getting the AI to do it even somewhat consistently requires a lot of tuning and guardrails / processing.

At some point I couldn't fiddle with the style / pipeline anymore and just had to roll with "looks ok to me" - the whole point of the project wasn't to faithfully adhere to a style but to push the limits of new technology and learn a lot along the way

Author here, and to reiterate another reply - all of the critique of "pixel art" is completely fair. Aesthetically and philosophically, what AI does for "pixel art" is very off. And once you see the AI you can't really unsee it.

But I didn't want to call it a "SimCity" map, though that's really the vibe/inspiration I wanted to capture, because that implies a lot of other things, so I used the term "pixel art" even though I figured it might get a lot of (valid) pushback...

In general, labels and genres are really hard - "techno" to a deep head in Berlin is very different than "techno" to my cousins. This issue has always been fraught, because context and meaning and technique are all tangled up in these labels which are so important to some but so easily ignored by others. And those questions are even harder in the age of AI where the machine just gobbles up everything.

But regardless, it was a fun project! And to me at least it's better to just make cool ambitious things in good faith and understand that art by definition is meaningful and therefore makes people feel things from love to anger to disgust to fascination.

Hell yeah oxen.ai is awesome, made my life so much easier

Yeah this is all completely fair and I agree with all of it. Aesthetically and philosophically, what AI does for "pixel art" is very off. And once you see the "AI" you can't really unsee it.

But I didn't want to call it a "SimCity" map, though that's really the vibe/inspiration I wanted to capture, because that implies other things, so I used the term "pixel art" even though I knew it'd get a lot of (valid) pushback...

As with all things art, labels are really difficult and the context / meaning / technique is at once completely tied to genre but also completely irrelevant. Think about the label "techno" - the label is deeply meaningful and subtle to some and almost meaningless to others

simple is relative, could definitely be done, but until the models get a bit smarter and require less manual hand-holding it'd be a lot of grindy work

yeah I had to put it behind a worker to deal with the subdomain and various other subtle caching issues. All in all cloudflare is incredible, and Claude makes it actually quite easy to deal with all the ins and outs

Yeah I'll get the code out there soon - it's just very vibe-y right now, the repo is a bit of a mess since I never bothered to organize things. The secret sauce is really in the fine-tuning, can definitely get those datasets/models public on oxen.ai too

Yup back of the napkin is probably about there - also spent a fair bit on the oxen.ai fine-tuning service (worth every penny)... paint ain't free, so to speak

oh wow that's such a good/obvious idea, i'll see if I can whip it together tonight

that's right - it worked very nice, but the models to generate the "shore distance mask" for the water shader weren't reliable enough to automate, and I just couldn't justify sinking any more time into the project

Haha I had to throw in the towel at some point and Staten Island didn't make the cut. Sorry (not sorry)

Interestingly enough even the big guns couldn't reliably act as judges. I think there are a few reasons for that:

- the way they represent image tokens isn't conducive to this kind of task

- text-to-image space is actually quite finicky, it's basically impossible to describe to the model what trees ought to look like and have them "get it"

- there's no reliable way to few-shot prompt these models for image tasks yet (!!)

100% - I originally wanted to do that but when I realized how much manual work I'd have to do just to get the tiles generated I had to cut back on scope pretty hard.

I actually have a nice little water shader that renders waves on the water tiles via a "depth mask", but my fine-tunes for generating the shader mask weren't reliable enough and I'd spent far too much time on the project to justify going deeper. Maybe I'll try again when the next generation of smarter, cheaper models get released.

Yeah I actually did that quite a bit too. I didn't want to get too bogged down in the nitty gritty of the tiling algorithm because it's actually quite difficult to communicate via writing (which probably contributed to it being hard to get AI to implement).

The issue is that the overall style was not consistent from tile to tile, so you'd see some drift, particularly in the color - and you can see it in quite a few places on the map because of this.

Yeah it leaves a lot to be desired. Once you see the AI it's hard to unsee. I actually had a few other generation styles, more 8-bit like, that probably would have lended themselves better to actual pixel-art processing, but I opted to use this fine-tune and in for a penny in for a pound, so to speak...

We had our third kid in late November, and I worked sporadically on it over the following two months of paternity leave and holiday... If I had to bet, I'd say I put in well over 200 hours of work on it, the majority of that being manual auditing/driving of the generation process. If any AI model were reliable at checking the generated pixels, I could have automated this process, but they simply aren't there yet, so I had to do a lot more manual work than I'd anticipated.

All told I probably put in less than 20 hours of actual software engineering work, though, which consisted entirely of writing specs and iterating with various coding agents.

the classic "water/concrete" issue! There's probably a lot of those around the map - turns out, it's pretty hard to tell the difference between water and concrete/terrain in a lot of the satellite imagery that the image model was looking at to generate the pixel images!

I mean this pretty literally though - I'm not particularly interested in these questions. They've been discussed a ton by people way more qualified to discuss them, but I personally I feel like it's been pretty much the same conversation on loop for the last 5 years...

That's not to say they're not very important issues! They are, and I think it's reasonable to have strong opinions here because they cut to the core of how people exist in the world. I was a musician for my entire 20s - trust me that I deeply understand the precarity of art in the age of the internet, and I can deeply sympathize with people dealing with precarity in the age of AI.

But I also think it's worth being excited about the birth of a fundamentally new way of interacting with computers, and for me, at this phase in my life, that's what I want to write and think about.

Yup the adjustment was giving cloudflare 5 bucks :)

That’s exactly right - the fine tuned Qwen model was able to generate seamless pixels most of the time, but you can find lots of places around the map where it failed.

More interestingly, not even the biggest smartest image models can tell if a seam exists or not (likely due to the way they represent image tokens internally)

Really want to do SF next. Maybe the next gen of models will be reliable enough to automate it but this took WAY too much manual labor for a working man. I’ll get the code up soon if people wanna fork it!

Sorry about the hug of death - while I spent an embarassing amount of money on rented H100s, I couldn't be bothered to spend $5 for Cloudflare workers... Hope you all enjoy it, it should be back up now

Should be back online now!

Should be back after some help from Claude and some money to Cloudflare

This project is awesome, and I love that there are people who are driven enough to make something with so much craft, attention, and duration.

That being said I have three kids (one a newborn) - there's no possible way I could have done this in the before times!