trekhleb

Thank you.

The comparison and the reference to the karpathy/llm-council is available on GitHub https://github.com/trekhleb/yesbrainer

In short, Karpathy's llm-council is one fixed answer→rank→synthesize pass behind a local server you have to run, while Yes-Brainer is a zero-setup browser app with three deliberation structures — including a real multi-round debate (consensus mode)

The app is open-sourced here https://github.com/trekhleb/yesbrainer - feel free to check the sources

This article gave me another nudge towards running Claude in a Docker container.

I made a thin Docker container wrapper "claude-pod" recently for my personal usage here: https://github.com/trekhleb/claude-pod

However, I wasn't using it that often, just because of that additional friction of running Claude via `PORTS="3000 5173" claude-pod` instead of just `claude`, etc.

But now I have more motivation for the containerisation :D. Not a 100% defence from the potential glitches, though, but still something...

Nice and minimalistic

I played with similar approach in JavaScript and built a NanoNeuron https://github.com/trekhleb/nano-neuron (it is more verbose than Python though)

What I mean is training something like GPT-3 in a distributed manner using a large number of regular browsers or laptops with average WebGPU support/power and WebRTC for communication.

Does it even make sense to ask this? Is it reasonable or feasible?

I understand there are many nuances, such as the size and source of the training data, the size of the model (which would be too large for any browser to handle), network overhead, and the challenge of merging all the pieces together, among others. However, speculative calculations suggest that GPT-3 required around 3x10^22 FLOPs, which might (very speculatively) be equivalent to about 3,000 regular GPUs, each with an average performance of 6 TFLOPs, training it for ~30 days (which also sounds silly, I understand).

Of course, these are naive and highly speculative calculations that don’t account for whether it’s even possible to split the dataset, model, and training process into manageable pieces across such a setup.

But if this direction is not totally nonsensical, does it mean that even with a tremendous network overhead there is a huge potential for scaling (there are potentially a lot of laptops connected to the internet that potentially and voluntary could be used for training)?

Thanks for the feedback! WebGPT is good. Looks like it is a vanilla JS? I used TensorFlow.js to offload all the troubles of working with tensors, gradients, and WebGPU integration to it. Along with a possibility to train the model in the browser it also helped to keep the actual GPT code pretty concise (<300 lines). Hopefully it will make easier to learn the model architecture itself for those who’re interested.

It is a very visual and entertaining visualization, I love it.

It inspired me to experiment with a genetic algorithm in "Self-parking car evolution":

https://trekhleb.dev/self-parking-car-evolution/

Thanks for the feedback! It is already in the roadmap here https://feedback.okso.app/feedback/p/love-it-is-in-desperate...

No it is not open source for now

Person “R” breaks into the person’s “U” private property and kills part of the person’s “U” family. Could you give me an example of the properly worded “why” part that could justify person “R”?

I think if we would need to select the best advertisement method/channel it would be people who are in love with the product :)

Yes, training may happen pretty fast in 2D, but then applied to the 3D to final visualizations. But this is just an assumption. I haven’t tried this approach yet.

Currently all calculations are happening in user’s browsers - no sending to the backend.

Yeah, this one is fun :D Really nice catch!

Yeah, the GA is not the best option for self-driving tasks, agree.

The reason why I chose GA is because I wanted to play around with this algorithm at the first place. And only after that I’ve tried to come up with some artificial problem I could try to solve with it :)

I haven’t thought about the native version, it was fun trying to implement it for browser. To resolve the performance issue I would try next to switch to the 2D simulation engine, since we don’t use the height during the parking. Getting rid of the 3rd dimension, lights and complex geometry would increase the performance drastically I believe

Yeah, the linear model is too simple to generalise :)

That’s a good idea.

However, there is an issue right now (https://github.com/trekhleb/self-parking-car-evolution/issue...), that the cars are not “punished” for hitting another cars (they are allowed to create the road accidents). That’s why if both cars have hit another cars they may continue driving and approaching the parking lot (only approaching matters so far). That’s not good, agree. But the app is in proof-of-concept stage, so it has the issues like this.

That was my first approach actually.

The mental model is like this: if sensor says 4 - it means the obstacle is 4 meters away. If obstacle is far away, then sensor may say… hm… 5 meters? 10 meters? Infinity meters? So I went with something a bit higher than max sensor distance limit of 4 meters. And, for linear equation this didn’t work for me. Cars were straggling to learn.

So I’ve switched to another mental model: if sensors says 0 - it means we just turn the sensor of, the sensor is not important. Let’s say you want to learn how to drive forward if the obstacle is behind you. Then you don’t care about the side sensors, you may just cancel them with zero variables. And with this setup, the cars started to learn much faster.

I think the correct approach depends on the brain “model”. For linear equation, canceling the sensor with the zero value of the sensor.

But if you would manage to train the cars well with the different approach - it would be really interesting to try

Yes, you may press the "Restore Evolution" button (at the bottom of the screen) and then upload the pre-saved training checkpoints from here https://github.com/trekhleb/self-parking-car-evolution/tree/...

You may also press the "Restore Evolution" button and then press "Use demo checkpoint" to use some pre-trained data.

Yes, currently the simulation performance is one of the biggest issues (https://github.com/trekhleb/self-parking-car-evolution/issue...). You may try to check the “Performance boost” checkbox that simplifies the geometry. It should give you approx x1.5 performance increase. But even with x1.5 boost the performance is still an issue, yes

I've used Tesseract.js to recognise the https://** links from the camera input and to make them clickable.

First issue I've encountered was the text recognition performance. Depending on the camera input (if the image contained something that looked like the text or not) I've got 2-20+ seconds per 640x640px image for text recognition on iPhone X. Not so fast as you may see. But the recognition was pretty accurate though.

The performance, as expected, improves when the image size is getting smaller and the amount of text on the image is also smaller.

Since I did't want to recognise the whole text, but only the links, I've used the TensorFlow Object Detection model to quickly find the areas with the text http://**. Then, instead of recognising the whole image I needed to do it only for smaller parts of the image. This gave some improvements to the performance: from the variable 2-20 seconds per frame I've got more stable 0.5-1 seconds. Also not good, but several times faster.

I've described the challenges in more details here https://trekhleb.dev/blog/2020/printed-links-detection/. But to sum up, I had a good recognition quality with an arguable performance with Tesseract.js

Cool! I’m glad that the link was useful! Another alternative that I’ve been using and that I liked is https://sketch.io/sketchpad/. Also pretty good tool (online and free)

I made them in draw.io

I'm experimenting with Machine Learning (CNN, RNN, MLP) and TensorFlow in particular:

https://github.com/trekhleb/machine-learning-experiments

In the repository there are several experiments, each consists of Jupyter/Colab notebook (to see how a model was trained) and demo page (to see a model in action right in the browser).

For now I've created only 10 experiments (i.e. Digits Recognition, Object Detection, Image Classification, "Write like a Shakespeare", etc.). But the plan is to do some more experimentations with GANs and RNNs.

The physics is really nice! Played with it for couple of minutes and even managed to drift/draw "2020" trace on the ground :D

Thanks!

Thanks! Glad that example was useful!

It is possible. But don't have strict plans yet.

Sure. Done.