typon

I am using a model that runs on a 100 GB300s, near AGI, all bets are off, $10B training run on a 1GW cluster, and it can't realize that when I told it to "please implement v2 of feature X" that I mean delete v1, not support v1 and v2 together, in a weird Frankenstein's monster of the two. Sorry, but I think my job is quite secure for the forseeable future.

If there really was a "simple" solution to Fermat's Last Theorem, Andrew Wiles wouldn't have achieved the important result he did, ending up making connections across disparate fields of math.

The LLMs "sweeping up" easy, or previously missed, results seems like a net negative. It's probably better for humans to struggle and come up with new tools than to just "clean up" low hanging fruit that doesn't add much value to the field.

Yes that's called Mythos 2 or GPT 6

There is ton of room for improvement "down there".

* Software inference optimizations

* Heavy quantization

* Chips with hardcoded transformer architecture

* Much cheaper HBM

* Much sparser models - 1T total with ~1-10B active params e.g.

* Not to mention - 2 years of today's frontier models writing RTL and kernels at superhuman levels.

By early 2028, major players like Intel, AMD, QC will ship accelerators in consumer laptops capable of running ~1T MoE models at ~100 tok/s

DinoV3 paper: https://arxiv.org/pdf/2508.10104#page=36

"we use a rough estimate of a total 9M GPU hours"

From CoreWeave, at current prices (~$2.46/hr spot to ~$6.16/hr on demand) would correspond to $22M–$55M.

The dataset is really where the cost is though - they used LVD-1689M - 1.6B images of curated web data from roughly 17B instagram images. This probably cost a huge amount of hours in human annotation, compute for algorithmic filtering, etc and not to mention probably a 20-50 person team working on this model.

You might want to change assumptions about how expensive these models are.

I remember saving up for a year to buy the ATI Radeon 9600 XT (I think it was $200 MSRP) so I could play the game on high settings. Now we can play it inside a virtual machine on a crappy laptop. What a journey

Always pains me to see all this innovation and cool software work being applies towards making a machine that extracts money from retail investors and makes a few people extremely wealthy. What a depressing and colossal waste of time.

Actually I think AI will largely automate software and math and really not much else in the short to medium term. (speaking as a computer/math person)

Next time there is a fire at your house I will say "he's an adult who should have been careful playing with dangerous things like fire, we shouldnt waste society's money and resources on saving his house"

That part should be non negotiable - the tech companies should be funding the energy infrastructure. My bet is that they'd happily do it. They are starved for space and energy, not capital.

Can someone explain to me why local governments are so against datacenters? It seems like a golden opportunity to build electric infrastructure that's paid for by corporations and if AI is a bubble at least that infrastructure will remain and continue to provide cheap power.

https://imgur.com/a/dV8gk3R

can you find curves like this for any other power source?

also batteries are getting exponentially cheap too

What is the point of building energy outside of solar farms? I'm sincerely asking

And the reason for _that_ is because of the callous way American society accepts the deaths of thousands of people who die due to the Healthcare Industrial complex (of which Brian Thompson was a key member of). Just because those deaths don't happen with guns doesn't make them any less important.

You will be forced to watch Firefly for eternity. Millenials will rule the internet for a 1000 years (a millenia).

Like an oasis in a desert of LLM slop. Thanks I enjoyed this README

We will be old, not boomers. Boomers are a special generation at a special moment in world history - they made decisions based on the limited amount of knowledge they had about how the world works and while some think those decisions have doomed us forever, I remain optimistic.

Even though it feels depressing right now, I think the post-boomer world is going to be an amazing place.

Is this why Google Search has been getting better and better every year?

The website design and content are much nicer than the "ideas" here. Just standard LLM slop once if you actually have read some of these books.

https://en.wikipedia.org/wiki/WPS_Office

500 million users. Just because you haven't heard it doesn't mean an alternative doesn't exist.

ideally you should get a deduction in later years

have you considered not all ideas are good and maybe they shouldn't see the light of day?

I often wonder whether neuroscience on LLMs is harder or humans?

This xkcd comic doesn't apply anymore due to AI making generating automation code trivial.

This is terrible

Can't wait till ChatGPT responses look like this

This looks amazing. I love tech in Lindy products that actually works well.

When you call yourself "Open"AI and then turn around and backstab the entire open community, its pretty hard to recover from that.