chrsw

China will accelerate on ML research and optimization regardless of US export control policy. They want to win or at least not lose just as much as the US and will pull every reasonable lever at their disposal to do so. NVIDIA and the US government have no say in this.

Google can't compete with China, neither can Meta. Only two labs in the US can keep chucking billions at the frontier race. Everyone else has a real business to run.

China can keep up because it's cheaper to run a frontier lab there. They also have more researchers and a stronger cultural inclination for this sort of thing. And I guess the business case in China doesn't have to work as well as it does in the US.

You don't want to replicate the exact model, you want to build a system of similar capabilities.

Correct. We need open weights, open code and open data. If nobody else can reproduce what someone did there will always be security questions. Even if we can reproduce it there could still be security concerns but it's more realistic to investigate yourself.

The compute constraints never mattered. If China had more compute they'd still end up winning because they have more people and a culture more inclined to math and science. Even if you find all this amusing, there's no own goal here. Not a policy one anyway.

Not just likely easier. Far easier. They're so far apart on any reasonable effort scale that the comparison is basically meaningless.

Science fiction has entertained and inspired millions of people and we should all be grateful for that but it has also distorted what people think space really is.

When you consider the scale of space it becomes pretty understandable why the Milky Way isn't teeming with civilizations sending large amounts of mass all over the galaxy. A realization one comes to despite the facts that it has taken humans a blink of an eye (on a galactic timescale) to go from tools to rockets and the Milky way is billions of years older than the entire history of the Earth.

In the US, there isn't one, which is why nobody in the US is currently doing it at frontier scale. And the people that were doing it stopped.

That's less than half TSMC's revenue in a quarter.

For better or worse, humans (or any animal) are a lot better at reacting than planning. I'm sure this technology will play out differently than any one of us, or any collection of us, can imagine. The possibility space is enormous.

Climate change is a political problem, not an informational problem.

My main takeaway from LeCun's thesis isn't that you can't build LLMs to do useful things better than the best human, it's that these systems don't learn arbitrary skills efficiently, like humans do. And the question is, why not? 8% on ARC-AGI-3 is amazing for a machine considering how far we've come since digital computers were first built. But it is pretty poor if you're claiming something is well on its way to exhibiting human-like intelligence.

Mythos can do some amazing things (I'm assuming, I've never seen it). A young child can learn to control its body without reading any books on dynamical systems and kinematics. Mythos cannot learn to control a humanoid robot after sucking in every piece of data Anthropic can get their hands on.

2024 is a good cut-off. I use these additional heuristics. Bad cover art. Not from a real publisher or not from an author that hasn't published before 2024. I hope that real books that just happen to slip through the cracks, that are also genuinely good books will eventually surface as easily findable somehow. Maybe that's wishful thinking, I don't know.

I know one thing. It's grim out there for publishing and real humans who have something good to say but don't have a popular voice.

Nice.

I wish there was something like this but that filtered out AI books.

I didn't mean nobody lives there. I meant if you plucked random people from all over the country and told them to relocate to NYC, odds are they would need a massive income increase to survive. And even then, it would be touch-and-go for most.

I'm from Harlem, and other parts of NYC. I moved out to pursue a career in an industry that is relatively non-existent in the NYC metro area. If I moved back now I could probably afford it on my current salary. But there are no jobs there for what I do. And if there was a bigger tech industry in NYC the costs would likely be even higher.

It's about $75k https://tinycorp.myshopify.com/products/tinybox-green-v2-wit...

It's not unusual, but most people can't afford to live in NYC.

Religion used to be the biggest supporter of science. Then science surpassed religion in civilization relevancy. Now conservative politics is stepping up to the fight.

how you guys handle the fact this leaks all your code and is stored forever on servers belonging to God knows who?

I don't.

My company doesn't host any code on GitHub, we have our own Git servers.

Regarding the privacy issues with OpenAI, Anthropic or anyone else, we engineers are just using the tools we've been authorized to use by the company. If there's a security issue, that would need to be worked out between the company and the model providers.

Would I use external model providers through an API for personal projects? For everything I'm doing now, probably. Could I see a day where I'm working on something too sensitive to be willing to give any data to these companies? Possibly.

I don't even look at benchmarks anymore. I just try different models as they're released on our large, proprietary, systems software codebases in real, shipping products or projects that will ship eventually. It's pretty clear which models help me do my job better or faster. I'm fortunate enough to have the token budget to use basically as much as I need, for now.

No need for benchmarks, evals, marketing, system cards or anything like that. I read the web for tips, practices and release announcements. My colleagues and I share our experiences with each other but beyond that, everything else is just noise.

In the broadest sense, I don't think we're there yet. I asked an SoC vendor to provide their chip documentation in Markdown. They refused. So, I went ahead and tried to do myself with AI.

I tried various AI tools and the results ranged from absolute garbage to something-but-not-something-but-not-quite.

I went ahead and did a section of a huge PDF by hand, just to see if what I was asking for was even feasible. After more than several hours of painstaking work spread across multiple days, I got several chapters to look identical to the source PDF in some Markdown renderers. I had to use some HTML for the more complex tables. I converted some diagrams to Markdown and some to images linked to from the Markdown.

Do they not use any memory chips in this thing?

It won't. Many people here in the US don't believe in science any more and they definitely think mRNA technology is some kind of conspiracy.

I know, my sarcasm was too subtle

Of course this can't go on forever. Especially not on LLMs. But are we really close to the limits of what these LLMs can do? I'm not sure we are.

The difference between GPT-5/Opus 4 and GPT-5.5/Opus 4.8 is striking. For software development anyway, there's no comparison. And all this has happened in a year.

My assumption is there will be another 2-3 years of improvements ahead of us on LLMs alone. Through hardware upgrades, larger training runs, better data quality, better algorithms, etc.

Of course, by then these models will be quite expensive. Will my company pay for it? I don't know. I'm sure some people will though.

They're not. And by the time they are Open AI and Anthropic will probably be onto the next thing.

Not sure what happened to Google in all this. They're falling out of the frontier race.

Not sure why this got downvoted. The amount of book slop on Amazon these days is staggering. It has completely changed my experience of shopping for books on Amazon and it's for the worse.

This feels like the Living+ Succession episode.

But they ran the numbers so there couldn't possibly be an agenda here. It's numbers.

Is this all vibe coded or human coded?