jaggederest

To concur it's "latency critical", not "performance critical", people often confuse those two - optimize it all, but especially the chained critical path latency!

but they are different.

How, and why?

We could very well end up where content IP is protected, LLM output is not and visa versa with reasonable legal founding, doubtful but plausible.

That is the current state of legal rulings - LLM output is public domain, not copyrightable.

I suspect that, in aggregate, all of the informational output of humanity prior to 2020 has taken more resources to produce than the last few years of LLM research.

I am sorry to be the bearer of bad news: That is just massive ACLs and permissions issues with extra steps.

This kind of god-of-the-gaps thing is kind of amusing to watch, especially since I can clearly see myself doing it over time.

Even if the models never get better than they are now, which is unlikely, they're already equipped to usurp taste, ethics, and everything else to a significant degree. Human feedback will always be valuable because we assign value to human feedback, but if you don't think an ethics-bot will ever be a thing, you're not seeing the current state of things.

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

Also notable, the internal list that this may or may not derive from: https://news.ycombinator.com/highlights

The issue is when users interact with claude, what can they see? If the AI can see super-secret-payroll-details.md - is it allowed to discuss that with any user? is there some sort of internal ACL within claude that asks pretty please don't tell everyone they're underpaid?

That's why multiuser agents can be an issue.

The other way to do this, is to have claude installed for everyone, and any user's chat with claude uses their personal permissions via something like oauth delegation or AWS IAM role kinda things. But that, again, is complicated ACL logic living through the delegation of roles and access

I am gently presuming that they are not selling tokens at a loss in their all-you-can-eat plans like the frontier models are, but maybe I'm wrong.

I am confused why they don't just offer a subscription with bring-your-own-harness, that would be a differentiator from claude/codex I think.

Really puts Disney in perspective. Imagine if there were a holding company running around suing people for referencing Catullus.

I struggle to reconcile this attitude with, among other things, the massive number of exploits being discovered and the pretty straightforward utility for coding. Have you used codex recently?

The real challenge is reigning in the size and complexity of the codebase if you're using AI to generate it. I have a bunch of skills (YAGNI / KISS inspired) but it still requires a significant amount of human effort. Knocked down about 9000 lines of cruft this month and I know there's another 10k in there sitting around.

This is a good/fast/cheap tradeoff, but in the form of accurate/precise/creative, or something in that ballpark anyway.

You can constrain the AI to be more accurate by having it a) equivocate (lose precision) or b) turning the temperature to effectively zero, thus losing "intelligence" and creativity.

There's no such thing as a free lunch. The models perform best out of the box, and everything you add into context makes them perform worse generally, but more likely to do what you want. Every token is a burden, but without those constraints the generation won't do anything.

So by that standard, hallucination is something completely core - you can't get rid of it without losing what makes them useful. You can make different tradeoffs, or use a very smart model in a way that is completely locked down (or adversarially verified, same result) which results in far more tokens used than if you just let the model do it's thing and validated for hallucination independently, outside of the generation loop.

Right, but I remember when spellcheck first started to be a thing, and people were like "Wow proofreaders are out of a job, ai is so cool", but then it became normal.

Right, and my point is that now that is the bar. If you'd told me in 2008 that markov chains would be smart enough to one-shot even a trivial video game from scratch I would certainly have called that AI even if you had to do some major rigamarole to get it to work.

That has ever been the case. As soon as it works reliably, it's not "AI" any more. Take spellcheckers or collaborative filtering as examples, but there are lots more. Hofstadter in G.E.B. said it well:

There is a related “Theorem” about progress in AI: once some mental function is programmed, people soon cease to consider it as an essential ingredient of “real thinking”. The ineluctable core of intelligence is always in that next thing which hasn’t yet been programmed. This “Theorem” was first proposed to me by Larry Tesler, so I call it Tesler’s Theorem: “AI is whatever hasn’t been done yet.”

Argentina on two steaks a day is quite possibly one of the best pieces of humorous writing, full stop. In my opinion at least, it should be regarded in company with Mark Twain and Jonathan Swift, though maybe not at the head of that particular table.

The blast clears that quickly in the atmosphere because the shockwave and debris move at-or-faster-than the speed of sound, so a few hundred meters is tenths of a second or less. You sit inside the mushroom cloud, of course, but the important part is gone quickly.

The Rapatronic camera was used to take these kinds of pictures, and you can see that the actual blast front is around 20 meters across after 1 millisecond (!!) https://en.wikipedia.org/wiki/Rapatronic_camera

I always describe it as an excellent 300 page novel crammed into 900 pages.

Good to see I'm probably not the only one who worked with a monstrous single database for all the things named "Yggdrasil", and a job server that was seemingly constantly trying to take it down named "Nidhogg".

I'm a big fan stylistically of what https://taalas.com/ is doing, as far as models baked into silicon. If you haven't tried their chat it's absurdly fast (and also very very dumb)

That implies to me that in the future we'll have models as good or perhaps better than the state of the art at the moment, but on hardware chips that can be put in places where you can't currently locate a datacenter, and operating at hundreds of times better power efficiency, which sounds pretty great.

The best part about LLMs is that, if you write gherkin style tests, you don't even need to automate them yet to get value, the AI can "manually" validate initially and then only codify them into deterministic execution after they've been nailed down. That way you have a bulk set of tests that run automatically, and a few candidate tests that are not automatically run yet, and the boundaries expand over time.

I honestly wonder about what percentage of the toxicity of society is a result of the awful copyright regime. Certainly another good chunk is a result of the patent system being broken.

I guess these are not the top items in the societal problems list, but they really don't help.

A fun example, always shake my head when I read it again: https://openai.com/index/where-the-goblins-came-from/

https://en.wikipedia.org/wiki/Ackermann_function

Yep! I also find that pretty frustrating, it's like, I got into programming because there are objective tests, now we're back to chanting at silicon like hedge wizards.

I often throw something into the prompt about how literal I am, and to never execute extended operations without explicit, concrete confirmation, but it's not especially effective.

Yes. Yes, good, let the hate flow through you.

We once telnetted into a different iMac (pairing stations) and set it to play a cricket sound every few minutes. Took us a minute to figure that one out.

And don't ask about Bear Force One and the unicorns.

Mr. Lerche actually went in and edited the unicorn "executable" for that one, before going on to merge Merb into Rails.

Capture the flag with AI is more fun than ever, in my opinion anyway. Rarely have we created a technology where sheer perverse enough mentality could break it, but today that door is open. Truly we are wizards whose incantations can cause superhuman intelligence conniption fits. Whether that's a wise idea...

Edit: also I've officially had AI damage hardware with that "wrong format wav" trick. $0.70 speaker was kaput.

The issue, at least as I see it, that they're trying to address is a pretty common one, where the AI tries to do whatever off the cuff suggestion, takes it way too seriously, and does something clearly unhinged. This kind of grounding, I suspect, makes it pull its head out of its metaphorical hindparts, and I suspect is a big part of the change from Opus 4.7 to 4.8 - it started questioning everything, they started injecting "wait" more, that kind of thing.

Also, the ultima ratio regum is "use the codebase to do something actually useful and report on whether it works or not", all code must intersect the real world at some point, and that's the point where the slop shows up.