openquery

Also the last generation to not frequent space. See the night sky up close.

The only thing the $1T model needs to do is find some algorithmic speedup which allows it to be trained at $100B. I'm not saying that's easy or that it will happen but I just don't see why not.

Unlimited intelligence doesn't mean unlimited resources or instantaneous implementation.

Of course you're right. At the end of the day you need to deal with the bedrock which is the laws of physics. I could be wrong but I struggle to believe we are close to the edge of what is possible in getting the most out of our limited resources or time.

Without atomic physics, uranium would just be another shiny rock in the ground. Sand is just what covers beaches. With enough time and intelligence we've made the shiny rock power cities and persuaded the sand to solve long-standing mathematical conjectures.

IMO Demis has the most reasonable takes.

LLMs show no sign of improving intelligence and instead providers are going down the ‘agentic’ rabbit hole.

I'm not sure where you're getting this. I don't work at Anthropic but Fable (Mythos) seems demonstrably smarter than Opus for pretty much any definition of smarter and they claim that Opus was used heavily in Mythos development (yeah I know take this with a massive pinch of salt).

Either way if the models are indeed helping development, even on the engineering, you can iterate on models faster and even if they're not contributing to core research yet you still have a baby exponential by improving the engineering.

I agree with your sentiment (about the noise), however I think this over simplifies it a bit. We may get AI that is super-human at frontier research and dramatically accelerates the pace, and still have to wait decades before it disrupts the job market (or maybe never displaces all work).

I don't see why that's the case when you have super-human researchers on tap. There are indeed physical (supply chain-y) issues to deal with but isn't the whole point that: 1. Super-human at AI research + scaling to millions of instances will probably result in super-intelligence in everything which is not AI research. (a subset of which is white-collar work) 2. Use that super-intelligence to solve any supply-chain issues you might be facing.

And we may just find that the human mind is way more capable than we thought and even with accelerating research it's just a harder problem than anyone expected, even algorithmically.

I hope so but whenever I do, I feel like I'm coping hard and not dealing with the facts.

I'm not saying we're there yet - I'm saying the trend lines are clear.

More likely (without careful vetting by the folks aggregating these models) is that the quality will go down as more and more AI-generated output gets subsumed into these models.

This assumes that there aren't algorithmic breakthroughs which reduce training/inference costs by several OOMs.

How much do these models need to do before people throw their hands in the air and say, ok this is happening. The Erdos unit distance problem, which as far as I understand was approached by multiple competent mathematicians was solved by a frontier model. Sure people argue there was no novelty there (I cannot comment as a non-mathematician) but it feels like they can draw lines laterally from deep knowledge in different fields (in this case combinatorics and algebraic number theory I believe) and solve problems.

Now if you have millions of instances running in parallel, all "probabilistic", working on frontier AI research I really don't see the blocker (and believe me I wish I did).

This is all noise. The leaders of these companies are flip-flopping to whatever sounds best for their current agenda - hiring, fundraising, pre-IPO, etc.

The only thing that matters is if LLMs with sufficient scaling can become frontier AI researchers kicking off the exponential. Everything else is transient noise.

https://archive.is/Pn7GU

Getting a company to IPO is not about getting lucky once - it's making hundreds of good decisions daily. From hiring to product direction to fundraising.

Like or hate the guy you gotta give credit where it's due. (Same for Musk, etc.)

The title made me think there was a cargo subcommand `cargo copy`.

For 99% of people I don't see the usecase (except for privacy but that ship sailed a decade ago for the aforementioned 99%). If the argument is inference offline - the modern computing experience is basically all done through the browser anyway so I don't buy it.

GPUs for video games where you need low latency makes sense. Nvidia GeForce Now works but not for any serious gaming. But when it comes to LLMs at least, the 100ms latency between you and the Gemini API or whichever provider you use is negligible compared to the inference time.

What am I missing?

I agree in general it isn't. But in this case Musk claimed that was the point of open-sourcing the algorithm. Transparency on what they are or are not suppressing.

I've always wondered - how can I as a non X engineer be sure that the code on GH is actually deployed on their servers?

Close to the end of the paper they casually drop that they solved all the millennium problems including the Riemann Hypothesis and have Lean proofs for all of them.

I don't know enough physics or mathematics to be able to tell whether any of this holds or if it's o3 generated engagement bait.

On further inspection the author seems to have a bored ape as their X profile pic so I'm going with engagement bait. Pretty high quality bait though.

Neither are containers.

This is demonstrably false.

You need a toolchain to compile to wasm and a runtime to run it.

I haven't played around with hypervisors much but the whole point of k8s is not just isolation but all the primitives the control plane gives you which you don't need to implement. Things like StatefulSet, ReplicaSet, Volumes, HorizontalPodAutoscaler, Service, DNS, ConfigMaps, Secrets, Accounts, Roles, Permissions etc.

Also the container runtime which is containerd by default I believe can be switched out for micro vms like Firecracker (never done this though - not sure how painful it is).

This is really interesting and a thorough write up. Thanks to the author for sharing their work.

Whenever I read about super low-level optimisation, my immediate feeling is that of gratitude to the author for spending so much time shaving off nanoseconds which the entire SE community gets to enjoy.

I wonder how much time humanity has collectively saved simply as a result of how all these optimisations stack up.

Thanks for the feedback.

Honestly sets as trees isn't original. While I was learning about ZFC I came across some lectures[0] by Richard Borcherds which was the seed of insipiration for this project.

[0] https://youtu.be/oWN13ktp8gg?t=1154

2 = {0, 1} = {Ø,{Ø}} by the Von Neumann ordinal definition.

Author here. Wasn't expecting to see this on the front page!

I'm really very far from a mathematician and this was a write up of a fun side project. I think the title would be unforgivably misleading in a formal context (if this was a paper claiming any new insights) but really it was a fun side project I wanted to right about. Maybe you read this and learned a little bit about set theory if you had no idea what it was (much like myself).

In general I resent popular science (especially in theoretical physics) which tries to reduce deep and interesting topics to poorly thought out analogies - but again my positioning here is not to educate per se. Or Michio Kaku style orating which assumes string theory a priori and later you have conversations with people who think string theory is established and tested because they watched a 40 minute video of him on YT.

Having said all this I need to get better and giving titles to the things I write - my other post about trying to build AGI in Rust got similar criticism.

Either way thanks for the feedback!

The representation of n has 2^n vertices. 2^2039484 might be a bit too large to render...

Thank you. I hope you write up findings from your own research.

To your point, I wasn't pretending that this work is novel or something that the AI community should take seriously. If anything, my point was that you can just do things.

I also feel like in the SWE community folks are generally concerned that LLMs are getting considerably better at doing our jobs. This was a poetic attempt at trying to regain some agency and not just let life happen _to_ you.

Were the brains event-driven? How did you implement the GA? What did individual genes encode?

Thanks for the kind words and the recommendations. Ordered!

By the nature of the math, if you can only use 4 of your 128 cores 50% of the time, your performance just tanks no matter how fast you're going the other 50% of the time.

I'm not sure I understand this point. If you're using a work-stealing threadpool servicing tasks in your actor model there's no reason you shouldn't get ~100% CPU utilisation provided you are driving the input hard enough (i.e. sampling often from your inputs).

Nice. Interested to see where this leads.

The network in the article doesn't have explicit layers. It's a graph which is initialised with a completely random connectivity matrix. The inputs and outputs are also wired randomly in the beginning (an input could be connected to a neuron which is also connected to an output for example, or the input could be connected to a neuron which has no post-synaptic neurons).

It was the job of the optimisation algorithm to figure out the graph topology over training.

Thanks for sharing. I thought the discontinuous nature of the SNN made it non-differentiable and therefore unsuitable for SGD and backprop.

Excalidraw[0] and a mouse and a few failed attempts :)

[0] https://excalidraw.com/