mbowcut2

I made myself reason it out, and came up with the exact same intuition. You need a sequence of 2n moves (n down moves, n over moves), but the sequence is completely determined by which moves are down (the others must be over). So it's 2n (moves) choose n (over).

Totally agree. Granting exemptions feels like trying to have their cake and eat it too. If regulations mean anything they need to be enforced so we can see the real downstream effects.

Gotta hit that docker system prune -a

Loved him in Secondhand Lions.

If you thought we were getting bad bugs before, just wait until the 90% agent-coded PRs start landing. We're gonna have multiple crowdstrike-level blowups.

It's an interesting concept, but I'm skeptical about how feasible this is. How much design/legwork/intervention will Seth actually contribute during the entire process? I'm thinking "growing corn" might be a little hard for a proof of concept, specifically because the time horizon is quite long. Something a little more short term like: contracting a landscaping job. The model comes up with design ideas, contacts landscapers, gets bids, accepts a bid. Seth could tell the model that he's it's agent, available to sign for things, walk people through the property, etc, but will make no decisions, and is only reachable by email or text.

Wow, I didn't know about the "skills" feature, but with that as context isn't this attack strategy obvious? Running an unverified skill in Cowork is akin to running unverified code on your machine. The next super-genius attack vector will be something like: Claude Cowork deletes sytem32 when you give it root access and run the skill "brick_my_machine" /s.

It makes me wonder about the gaps in evaluating LLMs by benchmarks. There almost certainly is overfitting happening which could degrade other use cases. "In practice" evaluation is what inspired the Chatbot Arena right? But then people realized that Chatbot arena over-prioritizes formatting, and maybe sycophancy(?). Makes you wonder what the best evaluation would be. We probably need lots more task-specific models. That's seemed to be fruitful for improved coding.

Seems like the less sexy headline is just something about the sample size needed for LLM fact encoding That's honestly a more interesting angle to me: How many instances of data X needs to be in the training data for the LLM to properly encode it? Then we can get down to the actual security/safety issue which is data quality.

I'm not surprised. People really thought the models just kept getting better and better?

it looks like the 2nd and 3rd bar never got updated from the dummy data placeholders lol.

It's not a new problem (for individuals), though perhaps at an unprecedented scale (so, maybe a new problem for civilization). I'm sure there were black smiths that felt they had lost their meaning when they were replaced by industrial manufacturing.

I've had similar experiences with vanilla ChatGPT as a DM but I bet with clever prompt engineering and context window management you could solve or at least dramatically improve the experience. For example, you could have the model execute a planning step before your session in which it generates a plot outline, character list, story tree, etc. which could then be used for reference during the game session.

One problem that would probably still linger is model agreeableness, i.e. despite preparation, models have a tendency to say yes to whatever you ask for, and everybody knows a good DM needs to know when to say no.

You can, and there has been some interesting work done with it. The technique is called LogitLens, and basically you pass intermediate embeddings through the LMHead to get logits corresponding to tokens. In this paper they use it to investigate whether LLMs have a language bias, i.e. does GPT "think" in English? https://arxiv.org/pdf/2408.10811

One problem with this technique is that the model wasn't trained with intermediate layers being mapped to logits in the first place, so it's not clear why the LMHead should be able to map them to anything sensible. But alas, like everything in DL research, they threw science at the wall and a bit stuck.

The problem with embeddings is that they're basically inscrutable to anything but the model itself. It's true that they must encode the semantic meaning of the input sequence, but the learning process compresses it to the point that only the model's learned decoder head knows what to do with it. Anthropic's developed interpretable internal features for Sonnet 3 [1], but from what I understand that requires somewhat expensive parallel training of a network whose sole purpose is attempt to disentangle LLM hidden layer activations.

[1] https://transformer-circuits.pub/2024/scaling-monosemanticit...

LLMs are better at LaTeX than humans. ChatGPT often writes LaTeX responses.

I think I agree with you. My only rebuttal would be it's this kind of thinking that's kept any leading players form trying other architectures in the first place. As far as I know, SOTA for SSM's just doesn't suggest significant enough potential upsides warrant significant R&D. Not compared to the tried and true established LLM methods. The decision might be something like: "Pay X to train a competitive LLM" vs "Pay 2X to MAYBE train a competitive SSM".

I read this as "pirate space industry" and got real excited.

It's interesting how I couldn't tell whether the rocket was 1m tall or 10m tall in this video. Turns out it's actually 6m tall per the link.

Nah, I think they made it model agnostic, which is kinda smart.

To a topologist, everything is topology.

Pack it up boys, they finally made the killer app.

I had a smart TV that gradually got slower and slower until it became basically useless. I figured it was just running out of RAM as apps got larger with updates over the years.

So, is this just an example of the first-mover disadvantage (or maybe the problem of producing public goods?). The first AI models were orders of magnitude more expensive to create, but now that they're here we can, with techniques like distillation, replicate them at a fraction of the cost. I am not really literate in the law but weren't patents invented to solve problems like this?

Me thinks he doth protest too much.

I agree that the moats are weak for applications, but I think there are possible strategies to capture users. One way is to make it difficult to switch, similar to Apple Music vs Spotify or iPhone vs Android. Although these platforms offer very similar features, switching has high friction. As an example, I can imagine an AI app that has been adapted to your use cases, maybe it has a lot of useful information stored about you and your projects which makes it much more effective, and switching to some other app would mean you have to move all this data or possibly start over from zero.

DeepSeek has demonstrated that there is no technical moat. Model training costs are plummeting, and the margins for APIs will just get slimmer. Plus model capabilities are plateauing. Once model improvement slows down enough, seems to me like the battle is to be fought in the application layer. Whoever can make the killer app will capture the market.

The vast majority of our work is already automated to the point where most non-manual workers are paid for the formulation of problems (with people), social alignment in their solutions, ownership of decision-making / risk, action under risk, and so on.

I agree. That's why I think the next step is automating trivial physical tasks, i.e. robotics, not automating nontrivial knowledge tasks.

oh, I think the horror element is there.

Absolutely horrifying. Well done.