overgard

You could have gained that stuff prior to LLMs. The leg up you're describing is free information on the internet, not AI. AI just makes it a little easier to find, while also crushing the original sources in the process. (Even if it had a broken culture, is stack overflow even going to exist in a year? Where are they going to train on going forward?)

I think it depends on where you think we are on the S curve of intelligence growth. (Yes, I think it's an S curve, not an unbounded exponential). If you think we're near the peak than playing catch up (especially if you can play catch up quickly) is very rational.

I know this isn't exactly a scientific test, but I had a local Qwen 3.6 27B model implement a fairly sizable feature today. There were a couple of bugs, mostly around me not giving sufficient specifications, but they were ironed out quickly when I pointed it out. I was able to ask the model to create instructions so next time it doesn't fall into the same pitfalls, and it did a great job. 27B local model! (And it was super fast too).

I ran Fable 5 as a code review and it didn't really have any significant corrections.

I guess my point here is that, for most work the frontier models are probably overkill anyway, and improving on overkill in a way that raises prices significantly is probably not a winning strategy.

The only place I can think of where the super high powered models are "required" is if you want to do a ridiculous token burn like GasTown where you just have it run un-monitored on very long tasks. To me though, that's an experiment, not a real workflow. And the way these labs are like "oh we made this (broken) thing in a week using just agents!" always also follows with "and it cost $100,000+ in tokens!". Like, ok, I get it if you're doing research but that's the salary of an entire person.. that can actually learn and improve.

It's both acceptable and inevitable. Play stupid games, win stupid prizes. This is what they deserve for the awful way they treat creators and creative people's intellectual property and livelihood.

Sad news, I grew up reading his column.

I really like this, but I think I'd be confused if I saw this without context -- the current behavior is so ingrained in me that any deviation would kind of be a wtf. Not unrecoverable but I think it's hard to change something so widespread.

Are there really downsides to creatine though? (Genuine question). All the advice I've seen makes it seem like it's basically just good for you even if it doesn't accomplish some specific thing. The only downside I've heard is maybe water retention?

These guys are not creators or inventors. They're hype men.

I'm fairly certain they're both malicious and stupid.

I think all that regulation will do at this point is help the incumbents who are failing. Protectionism. I don't think they deserve that help. I also don't see any reason to think the current administration would have anything resembling competence around this. And it's worth noting that Greg Brockman is a huge MAGA donor, so it's likely the policies would be very corrupt. (Don't worry, he justified his donations as "apolitical", he just wants to buy the politicians, he doesn't believe in their causes. I hate these people.)

I don't trust these people, this reads 100% like PR BS.

Oh cool, you're remaking things that already exist. You know why you can few shot those things? Because they already exist.

One thing that I find frustrating/disturbing about LLM generated code is that wrong stuff doesn't usually look wrong, so you almost have to review it even closer than what you'd review from a person. Like in the past when a developer was being sloppy, usually you could kind of tell superficially, but with LLM code it's usually well documented and superficially well structured, all while doing batshit insane things. Like today I noticed some code that it had written was just silently returning on errors without logging a warning or returning an error code. It "worked" but it was a major foot-gun!

Yeah, it used to be the friction of making something was at least a partial sign that it had been thought out to some degree. Now it's just "lets lob it on users and make them figure out if this is worth a damn". I feel like with a lot of this stuff, there isn't much value to being an early adopter and there's a lot of risk. Better just to wait a couple months to see how things shake out. If it has staying power it really won't matter if you're behind by like a few months.

I'm kind of getting to the point where if I know something is vibe coded, I just won't use it. It's not an anti-AI thing, it's just a quality thing. Pretty much every piece of vibe coded software I've used has been bad in some regard. The worst ones are the ones that aren't obviously bad but rather subtly bad in dangerous ways (the article about OpenCode yesterday definitely made me nope out of using that)

I imagine they'd be split up, and the essential database business would be what's actually bailed out.

Cal Newport has a grounded take on what the Erdos thing meant in practical terms: https://youtu.be/fhZRWZ6J4k4

Long story short this is much less impressive than it was sold to be. Basically they had mathematicians combing through long winding chains of thought (incidentally: you wouldn't have access to that reasoning) and cleaning it up and making it coherent. That doesn't mean its unimportant, but we're being gaslit about the amount of human steering and human effort that went into this.

A plea: please stop upvoting hype that comes from these labs. It takes time to evaluate their claims and they're always less impressive than claimed.

Good news, the new manual will be 83500 pages.

Of course it is

I think stylized scenes can be just as immersive. I've never been in a game and been like "GAH this pixel art is throwing me out of it!" I'm not saying realism is without value, just that it's probably a little overrated in the graphics space.

I agree, although I think it will be a lot sooner than 10-15 years. I'm running local AI right now and it's definitely not production grade yet, but it's surprisingly good. Speculative prediction that I probably shouldn't make: when the bubble pops, depending on when it pops, RAM prices might drop a lot. I could foresee these companies having produced a lot of RAM that suddenly doesn't have a buyer. (I know high bandwidth memory is different, but I imagine there are companies that will want to take advantage of that)

'Libre Office' did not 'win'.

I think France and other parts of the EU are switching over to it. Although that's probably more due to Microsoft's aggressive behavior recently. Agree on the cost thing generally, but I can't help but think that when the hype to "do AI" blows over, people are going to be casting a jaundiced eye towards data security, which probably means self-hosting and sandboxing

That seems hypothetically possible, but kind of hard to do? I don't know a lot about training, but it seems like they wouldn't have exact control of the data that goes in at that scale (scraping the internet) so it seems like it would be kind of hard to do that in a way that's subtle. I'm kind of reminded of Elon Musk trying to put his political views into Grok and it seemed like it created huge technical problems with the model saying some really out of control things. Maybe it was just an xAI issue though.

The default system prompt is opinionated (fine) but it has shit opinions (not fine). It took me a while to figure out why my agent kept saying “Use ABSOLUTELY NO COMMENTS” when dispatching subagents.

GAH!!! So that's why its been deleting comments. That is annoying as hell.

FWIW though, I think this applies to other harnesses also (not the annoying bits, but the security risks). I was thinking this morning about how much of a supply-chain vulnerability these tools are. Like, basically, they have a ton of data, they're updated practically daily, and given their vibe coded nature I really doubt anyone is auditing the 10,000 NPM dependencies they drag in. It really will just take one left-pad incident for this to be an absolute disaster.

I do think open-weights models are going to "win" in the sense that they're probably going to be dominant when the hardware to run them becomes affordable. (which might be a while). Although I guess you could probably rent the GPU's yourself to hypothetically save on costs. (I'm a little skeptical -- I've heard of companies doing this and the inference bills are surprisingly high -- assuming the sources are correct. I don't know if a lot of people really want to be advertising "oh god our bill is horrible")

I'm sort of baffled by what the entities that train the open-weights models get out of it though. Is it just a direct play to undercut the US providers because they view them as a threat? I just don't really understand the business model behind it.

This isn't quite true: https://www.congress.gov/crs-product/LSB10922

The clearest part from the page:

Before the proliferation of generative AI, U.S. courts did not extend copyright protection to various nonhuman authors, holding that a monkey who took photos of himself lacked standing to sue under the Copyright Act; that human authorship was required to copyright a book purportedly inspired by celestial beings; and that a living garden could not be copyrighted. The U.S. Copyright Office has also long maintained that copyrighted works must be "created by a human being" and therefore refused to register works that are "produced by a machine or mere mechanical process that operates randomly or automatically without any creative input or intervention from a human author."

I think the long story short is if you're incorporting LLM generated content into your own work, then your own work is copyrighted but if it's purely AI created and you never put any of your own work into it, it's not (so: think the slop cannons people are pointing at social media for example -- probably not copyrighted (although I don't know why you'd want to put your name on it anyway)). Granted, I don't know how you'd ever prove that something was purely LLM generated so it's kind of a technicality for coding, but for image and music I think it applies pretty clearly.

As an aside, I think socially maintaining that copyrights are for humans only is extremely important. It might be the most important legal matter around this. I see nothing good of allowing AI labs that have strip-mined the internet to claim a copyright on the things they regurgitate, and I don't think spammers deserve copyright protection either. The human-only requirement is a very good one.

So, code is not the only data they potentially have access to, and it might not even be the most interesting data. There's also whatever other things you plug the AI into: slack, email, internal documentation, etc.

Also, a lot of people using LLMs are using it on code bases that were written before LLMs were common for coding. So in that context, you really are allowing them to train on your code for free.

I keep thinking about the Figma thing. If you're unaware, here's the google summary:

----

The Board Departure: Mike Krieger, Anthropic’s CPO and a co-founder of Instagram, sat on Figma’s board of directors. He resigned on April 14, just days before news of Claude Design broke. This sparked speculation over conflict of interest and the use of proprietary product strategy information.

Betrayal of Partnership: The launch aggravated the tech industry because Figma relied on Anthropic's models to power its own AI features, and even announced a joint "Code to Canvas" integration. Reports indicate Figma was blindsided by the depth and scope of Claude Design.

Market Reaction: The "SaaSpocalypse" thesis—fears that major AI foundation models will rapidly build application layers and cannibalize their own SaaS partners—was realized when the news broke. Figma’s stock saw an immediate 7% drop upon the announcement.

----

I would suggest to people using LLMs: you should be cautious about giving these companies data or relying on them. If you're building an AI startup, there's a very good chance they could decide to directly compete with you if your idea has traction. You're also at their mercy for API pricing etc.

I agree with his overall point, but I would point out: realism is not usually the point, the point is to look good. The problem with realism is once you get past "wow the computer can do that now", it's not very interesting -- most of reality is pretty dull looking!

Woof, I thought you might be exaggerating but I priced it out and you're right, it's actually way over $8000 now. I hate how the AI industry is making everything more expensive for everyone.

Also, offer a paid course