sebosp

I am working on my project swarmy, making data analysis of Starcraft 2 replays easily available for gamers/casters, all in rust, using nom to parse replay binaries and map data, exporting to apache arrow snapshots to analyze with polars, managing the data flow via tauri, exposing the aggregations/controls via leptos, using rerun to navigate replay timeline, bevy to deep dive through the maps and provide "geo" stats, this weekend started playing with bevy skein to manage assets/materials via blender.

I made something like this around 10 years ago to show wireless frequency coverage, luckily the country I was living in is small enough and close to the equator, I didn't support the curvature of the earth so I can't start to imagine how difficult it must have been to visualize it in the browser https://sourceforge.net/projects/waire/ in my case the earth was kinda flat lol, it was fun parsing the map data available back then and playing with SVGs, back then the rendering engine from Opera browser was unmatched

Wow this is super cool! Will you use something to draw this on a map? If so what? Is it the same data format available in Europe and North America? What is something you suspected was going to be trivial and turned out to be extremely difficult?

Awesome, maybe some negative effect over it, how often does it refresh?

Can you think of a way to use your live cloud data as the background shader for https://github.com/sebosp/chartacritty ? I was also hoping to draw the iss progression through the earth, but sometimes people use terminals to actually read and may become confusing... For the rewind functionality you could look into rerun.io, not sure it's compatible with your current libraries but may be fun

What's the equivalent to foreman/quickstart but for embedded devices, ie you got a bunch of devices, different architectures, different sensors to different pins, different versions of firmware, some of them off, how do you keep track of them?

Would this fit your medium to long term? It's a weekend work to automate: json logs go to Kafka, logstash consumer to store batches in hive partitioned data in s3 with gzip compression, Athena tables over these s3 prefixes and prestodb language used to query/cast/aggregate the data

This reminds me of MIT's open courseware, 601 SC, unit 1 with state machines, going all the way to build Fibonacci with them without recursion, tbh the moment the teacher translated the state machine to bounds to electrical circuit I felt it was a leap and I couldn't quite understand their relationship, maybe I missed a requirement course. I tried to express that course in Rust as one of my first projects learning the language here https://github.com/sebosp/rustexercises/blob/develop/ocw601s... and I think a similar iteration in the direction of this project would be to build the dependencies as drag-and-drop boxes over the browser (maybe egui) and connect the state machines by clicks, maybe download the generated code as either bash or compilable rust code, you know, for kids.

Now I feel I have to read again Haroun and the Sea of Stories from Rushdie thanks to your last comment <3 that's one of my favorite childhood reads

I'm old and I sympathize with your last sentence, I _imagine_ certain reinforced paths through repeated experience are myelin covered so much as to be almost static. But if I think of generations growing up, their brains aren't reinforced/seasoned enough to spot these "Wait... What?" moments in online content that is almost sensical but not quite. In a world with ever increasing AI hallucinated content, when children absorb fake content and build strong paths in their brains, at some point the words/meaning could be so distorted that you cannot understand anymore the person in front of you? I think of the political landscape where we have similar problems, social media algorithms curate content to keep you entertained within subjects A and B, and you have a community with shared values and you are seasoned and share its language domain. How would the landscape look like when the net can be bombarded by content that appears to be true, valid and useful but it really is not, for young generations? Also if somebody can paint the right myoline picture in my head (not plausible text generation but science) I would really appreciate it.

This is so worrying for me, the amount of digital garbage that can now be generated that is not obviously garbage, that one must read and discern first nonsensical bad text generation and second if factual, is truth becoming a needle in the hay stack? How can this be cleaned?

Reminds me of https://en.m.wikipedia.org/wiki/Altered_States

Is this from a book? If not please write it

Is there an approximation/ratio in which the amount of digital garbage/hallucinations online generated by AI is so big that it cannot be used to train AI itself? Like are AI companies running against the clock because, say, in 5 years the internet will be flooded by false information to such an extent that it would render the internet as an invalid training ground. In a way requiring a snapshot of the internet pre-AI, because this is click bait problem times infinity it feels like

I wonder if AI generated web pages block AI agents from indexing their content, like, one engine indexing the other's content in a loop until the amount of digital garbage is so gigantic that is the end of the information era, or how are we ever stopping this? What is our failsafe?

The bad: I was an alcoholic and smoked from age 12 to age 30. The community builds a level of arrogance that they have access to special knowledge because they are in a specific stage in the evolutionary "destiny". Breaking bonds with other family/friends is almost needed because you don't hear their points anymore, just like scientology or other closed sects, not sure their proper name. You think you have a guardian angel too that your destiny is to survive and become enlightened and you shouldn't have fear, I had a couple of near death experiences because of this false sense of security, plus the constant wine drinking... The good: they pack a lot of trigonometry and algebra but then they teach you "sacred geometry"... They also teach basic quantum mechanics basics, but then teach you that the observer effect is controlled by our minds, or that you can transform a carrot into gold... So somethings are useful... I'm still recovering from these cults, it's crazy how much our minds can change, therapy takes a while to take and forgetting/re-learning also takes its time, as it should...

Even seated you would make these psssshhht sounds of pushing air out, you would end up sweating from a meditation session because your muscles would also be contracted as you focus on the hand position in your chest... In your mind you have to visualize words backwards and breath them out letter by letter... I still laugh at the "so be it" that has to be said with absolute certainty for your wishes to come true

Laugh all you want, when I was like 12 years old my family was into this weird techno-pseudoscience-cult of "Ramtha's school of enlightenment" and they have the biggest trance/shamanistic playlists, we had these long ceremonies of hours of dancing while listening to music like this and drinking red wine and smoking tobacco on pipes. I guess if it wasn't for the alcohol I would have remembered all these amazing songs but well... I think that sect is still alive and I wish they had a playlist somewhere outside their website coz I can't stand the website much due to the religious content bringing back drunk memories. If you know of such a playlist please share

I would love to learn about this... Specially in the era of demucs, maybe some python automation to ableton live tracks that we can modify through the day slightly? I have playlist with dozens of 1+ hour videos on YouTube of psytrance and I find them so enjoyable, full of bass and good for focusing (granted there's the obligatory 1 minute muttering in the song of Terrence McKenna but OK, if psilocybin helped them create these amazing songs then so be it )

How does ZMQ compare to recent message passing libraries? Does it have a place in rust's tokio ecosystem? Are there projects that implement the HA transaction store to disk protocol "out of the box" ? Forgot the name... Binary star something? I always think of working with ZMQ as I really enjoyed it, but I can't see it as easy no deploy as Kafka with so much ACL tooling, easy Auth and debugging, dashboards, metrics... Maybe one more question... What are recent projects that use ZMQ behind the scene that you find a good place to learn more message passing techniques and their tradeoffs?

This. I have read the guide a few times through the years, it's such an interesting dive, specially if one follows the programming exercises... I must admit that any time I work with Tokio or other message passing libraries I end up doing some sort of majordomo, I do not know if this is food, but MDP got stuck in my mind for better or for worse... For example with Tokio mpsc I end up creating majordomo coordinator, clients and services in separate threads, I don't know if this good tbh but it's just so easy to split (and so easy to shoot myself in the foot with synchronize shutdown across threads). Do you know of other guides like this?

Is there a way to get a replay pack with order of millions of games (maybe more?) ? I'd like that to be able to create meaningful Stats, proper data analysis, training, etc... Wouldn't mind paying a bit for that... So far for the exploratory data analysis I've done I only have my own games and 2 or 3 replays I've been sent that don't work...

Wow that is interesting that it patches back and forth, super cool! How did you find out about this? I wonder now and then if one can "consume" the events on a live game somehow, like connect as a bot or sniff the events before they are encrypted through the network... Also curious about the drone kill counter mod/overlay that Harstem uses, do you know of the name of that? I can't figure it out but would love to enrich it with stats from the data analysis

I only had one replay with direct strike in it's name but I can't remember watching it, I thought it was just one more map... I know it was failing initially to decode but in the end I fixed a couple of bugs and then it was working with this file. Could you explain what is significant go direct strike? How is it different to a normal game?

Indeed, another thing is that imho, casters and you tubers are data scientists that have the indexed dB in their heads, they say "at second X in this map Y you would have Z upgrade" and they talk about efficiency of mining, but I always wanted to see these things as statistics visually, like, how much faster or slower is player X in this game compared to the standard? How does the meta evolve? Or worse... If I check your replays, and you never ever scout this area, I could proxy there. You could find the build that in this map you never prepare for because let's be honest we are much more predictable than what we'd like to admit... I hope this project unlocks these kind of possibilities

Have you seen rerun.io? Also web assembly, sharing again the link from above https://sebosp.github.io/swarmy/public/0.5.1/2023-GSL-S1-RO1... Maybe you can use it too in your project! The guys from rerun are amazing and super helpful, had a call with them to discuss questions and possible uses too

Btw such move instructions I visualize here with the amazing rerun.io https://sebosp.github.io/swarmy/public/0.5.1/2023-GSL-S1-RO1...

This brings another question, if I watch an old replay, does it mean it has to have the versioned game engine stored in the computer? Does it mean with every update it gets increasingly bigger? Or is there a number of supported versions you could replay? I only started playing like 3 years ago...

I mean for example https://github.com/sebosp/s2protocol-rs/blob/755098fb86ab6b1... I hacked my way around the json protocol specification, an enum has n types, using log I can find the number of bits I need to read to uniquely identify each variant, that kind of serialization I wonder the name of.

I fantasize if training an AI to watch all the replays of a map on YouTube can create a repro of the state machine logic that moves the units through the game, or to find the parts of a map that cannot be walked upon. In the replay it emits an event when the user wants a unit to move to x, y but then it has to path its way there, and I cannot predict (without their logic) how it would move there