My Tamagotchi is an RL agent playing Slither.io

https://nkasmanoff.github.io/#/blog/tamagotchi-rl-slitherio

by nkaz123 • 7 months ago

38 18 7 months ago

nkasmanoff.github.io

Corporate firewall is blocking this since its a "newly registered domain" but I wanted to note that Tamagotchi got me to revisit Digimon after I learned that Digimon was created as a way for Bandai to sell to boys. Color me surprised when I learned that Tamagotchi was considered for girls, but I played with mine like there was no tomorrow, with the Pokemon hype of the late 90s it came to many of us at the right time.

Surprisingly, and I just looked it up, you can buy the original classic ones for about $20 straight off Amazon.

nkaz1234• 7 months ago

I recently updated my .github.io to route to a domain name I purchased so that could be why it's getting blocked right now.

This comment alone has more tamagotchi lore than my post as a disclaimer in case I saved you a read haha.

remix2000• 7 months ago

Clicked for Tamagotchi, but I saw none. My day is ruined. :'c

nkaz1234• 7 months ago

I'm sorry for the clickbait

gnyman• 7 months ago

It caught my eye also but the article was interesting so I'll forgive OP :-)

On the topic of tamagotchi, if you happen to have a flipper zero there is emulator for it :-) my kid enjoyed it for while and it saved me a few bucks from having to buy one.

https://github.com/GMMan/flipperzero-tamagotch-p1

You can run it on tama-p1 on other platforms also but the flipper was very reminiscent of the original one.

jeremysalwen• 7 months ago

As someone who implemented some RL algorithms and applied them to a real world game, (including all the ones mentioned in the article), I would be surprised if the implementation is not buggy. That is one of the most striking things about RL, the extent to which it is hard to find bugs, since they generally only degrade the performance instead of causing a crash or obviously wrong behavior. The fact that he doesn't mention a massive amount of time spent debugging, and the longish list of things that were tried that really should have worked but didn't, suggests to me it's probably still buggy. I suppose it is possible that LLMs could be particularly good at RL code since it's seen it repeated so many times... But I would be skeptical without hard evidence.

nkaz1234• 7 months ago

I accepted the bugginess in the browser game as unavoidable, and probably had too much faith in the LLM implementations, but I did a bit more troubleshooting than mentioned. The progressive improvement over episodes (and intuitively that PPO > the others) gave me some confidence, and I've since used a similar setup on 2048 with more results showing improvement over episode: https://wandb.ai/noahpunintended/2048-raspberry-pi?nw=nwuser...

treyd• 7 months ago

I had this idea during the pandemic 5 years ago now, and even did some of that work to figure out the variables I'd need to extract to make it work, but I never found the time/motivation to work on it for real. Really happy to see someone put in the effort.

breakds• 7 months ago

The sample efficiency of the RL algorithm, even for simple games, is not very good. This usually means that we will need a lot of episodes for the policy to learn to excel. Being able to run policy in an environment that can parallel and accelerate could be very helpful for the improvement - for example running a batch of browsers or tabs simultaneously :)

ohazi• 7 months ago

(Sent through Gemini to blur my monitor).

Excuse me, what?

Stefan-H• 7 months ago

The image is watermarked from gemini, so presumably the author was trying to allay concerns that the important content was fake.

idiotsecant• 7 months ago

That doesn't answer the question of why you would use an LLM to blue your monitor when there are a thousand ways to do it yourself

nkaz1234• 7 months ago

Yeah essentially this. The irony of about wanting to obscure information by submitting it to a model API isn't lost on me, but it was the easiest way I could think of. Wanted some way of making the most key content in my picture to be the only thing unblurred

p0w3n3d• 7 months ago

  Sent through Gemini to blur my monitor

Blurring is never the solution as it can be unblurred in most cases (look up Mr whirlwind). Also Gemini sounds like overkill for the task of burring in general. Inkscape and gimp can do it for free (providing that you have a computer, not an iPad for example)

My Tamagotchi is an RL agent playing Slither.io

Related Stories