viscanti

I believe the author's thesis is that if they had invested in innovation over a couple decades, the product probably would have sucked less.

They had pretty drastic price cuts on Opus 4.5. It's possible they're now selling inference at a loss to gain market share, or at least that their margins are much lower. Dario claims that all their previous models were profitable (even after accounting for research costs), but it's unclear that there's a path to keeping their previous margins and expanding revenue as fast or faster than their costs (each model has been substantially more expensive than the previous model).

Brand recognition that they're throwing away with a rebrand.

You're watching video podcasts while hiking or what's the weekend hike use case for more than 27 hours of video playback on a single charge?

Well how much of it is correlation vs causation. Does the next generation of model unlock another 10x usage? Or was Claude 3 "good enough" that it got traction from early adopters and Claude 4 is "good enough" that it's getting a lot of mid/late adopters using it for this generation? Presumably competitors get better and at cheaper prices (Anthropic charges a premium per token currently) as well.

It's more like the patient needs some fixed amount of food each day and it doesn't make a lot of sense to create lots more food than they need on the hopes that someday they'll want to eat more than they can.

If the argument is that everyone should focus on the arts at the expense of everything else, it's hard to imagine that's an ideal outcome relative to alternatives. If we're not arguing that everyone should focus 100% on the arts (no other degrees should be available), then it's a matter of degree and certainly some outcomes might end up with more people pursuing the arts than what society needs.

It's much more conservative in the scope of task it will attempt and it's much slower. You need to fire and forget several parallel tasks because you'll be waiting 10+ minutes before you get anything you can review and give feedback on.

If it's not astroturfing, the people who are so vocal about it act in a way that's nearly indistinguishable from it. I keep looking for concrete examples of use cases that show it's better, and everything seems to point back to "everyone is talking about it" or anecdotal examples that don't even provide any details about the problem that Gemini did well on and that other models all failed at.

A16Z

This kind of proves the point? Presumably your mother didn't buy the latest phone for "continuity" or camera improvements. The features and additional hardware improvements might be noticeable after being used, but are they driving sales to people who aren't tech enthusiasts?

If you're able to generate minified code from all the code you can find on the internet, you end up with a very large training set. Of course in some scenarios you won't know what the original variable names were, but you would expect to be able to get something very usable out of it. These things, where you can deterministically generate new and useful training data, you would expect to be used.

Because of how trivial that step is, it's likely pretty easy to just take lots of code and minify it. Then you have the training data you need to learn to generate full code from minified code. If your goal is to generate additional useful training data for your LLM, it could make sense to actually do that.

I can't understand the hate

I think it's because of the promises of the team (new Large Action Model) vs what's actually being delivered (the model is some scripts). The team has a history of over promising and underdelivering (or scamming - depending on your perspective). It's also economically unviable. Somehow you're meant to get free LLM calls for life but there's no way for them to actually cover those. There's not really any communication about how it might be a limited time thing for early adopters or how it could ever get to be sustainable.

If they had focused on what they have, they probably could have charged the same amount and people would generally be OK with it. But they've over promised and under delivered again. I think the reaction is pretty understandable.

It seems to be difficult to turn the pure research back into new products. Apple famously got lots of ideas for free from Xerox PARC. Google researchers wrote the Attention Is All You Need paper and they're now desperately playing catchup because they couldn't convert it to any kind of product. There's nothing wrong with companies investing in pure research, but these large companies sometimes are unable to take advantage of the research. The people running the business want to keep doing what got them successful, not some new experimental thing that might not work.

No one has ever brought a native (not 3rd party) calculator to the iPad before. Apple is the first.

On device or in an Apple owned DC. It sounds like they have aspirations for their own Apple owned LLM. ChatGPT seems like it's there until they can get something good enough to generally replace it for cases where their in-house solution isn't capable enough yet. They likely continue to invest heavily on big capable LLMs as well as ones that are small enough to run on device (while working on the hardware side to ensure they have the device capabilities to run more powerful models on the device).

They say it's going to be free forever with no subscription, but they have to pay for chatgpt API calls. Even if you forgive them for overhyping their chatgpt wrapper, they're still a ponzi scheme.

If you know a great deal about what is right and wrong, and you choose to do something bad, that feels worse than being bad and not knowing any better.

For some people, the comments are the worst part of YouTube. I could see them being pretty vocal about not liking the design that makes them more visible.

Well the unit economics don't make any sense. You get unlimited free LLM calls forever? I don't really see the argument for it shipping too soon (although it is too soon for what they're claiming). There's a basic unit economics problem that can't be solved regardless of their future roadmap. They're promising people a no subscription model, claiming it eventually will have a Large Action Model and LLM for question and answering, and somehow those will be available free forever? It's either a get-rich-quick scheme, which sounds likely given their background as crypto grifters, or complete incompetence (somehow no one there ever thought what it might cost to do inference). Neither seems like a legit company that just shipped too soon.

I wonder if China or other countries will do the same for US apps now.

China has been doing this for a very long time, to keep US apps out of the Chinese market (or forced IP transfers to Chinese companies) so that Chinese apps can thrive. There's no room for them to retaliate because they've already gone as far as they possibly could.

If it means that I get to work with well commented and tested code, that might be worth it.

Zuck renamed the company Meta, partially because of a deep believe in the Metaverse (perhaps it's too early to call it, but it seems misguided) and partially because of massive amounts of negative press they were getting because of pretty toxic decisions that he was ultimately responsible for at Facebook. I love that he's passionate about AI and that he's supporting the release of somewhat open source models. But it seems a stretch to call him the best right now.

OpenAI should make something so that people can enter their prompt and maybe even drop in a knowledge base and then share with anyone else who wants that functionality.

They didn't actually fire the board of the non-profit. They just said they'd all quit in protest because of an action of the board they all felt was egregious. The board could have stayed and been a non-profit that did nothing ever again. They decided it was better to step down.

AI will make it far easier to keep people in line.

An alternative explanation is that AI companies are using RLHF to help the 90% understand that a lot of things aren't black and white but that there is relevant context to keep in mind. For many cases, that's probably useful (like if someone is asking if one movie is better than another movie). At the extremes, which are hopefully rare (are a lot of people really wondering if Hitler is better or worse than some Twitter posts), you get nonsense.

But the base model, when its trained on the whole internet, will have some extreme biases on topics where there's a large and vocal group on one side and the other side is very silent. So RLHF is the attempt to correct for the biases on the internet.

Was this a "5 aces" level of poker hand? Or are we not quite there yet?

One drawback might be that political will and attention is fleeting; and interests become vested in a current state, blocking change; and it's now or not for another generation or so.

If a problem, like tracking what non SOTA models are doing with respect to safety, don't rise to being worth regulated, then why bother regulating it? If you're going to force everyone in the field, in California, to file additional paperwork (and understand what they're legally supposed to report and how and when), then you probably should have a compelling argument for why. If you didn't do that now, and it's never a big enough problem to regulate then you've done no real harm. If it's clearly an issue that needs to be regulated in the future then it should be easy to do so.

A lot of hand waiving about how "minor paperwork" for everyone isn't such a big deal. But very little coverage of what benefit that paperwork is accomplishing. This looks like a big bag of regulatory requirements, some which might be good and some which might be bad ideas. If it's the start of a discussion around what actually good regulations look like, then they have to start somewhere and maybe it will accomplish that. If it's an actual proposal and we have to take or leave everything, maybe they need to go back to the drawing board and figure out something better.