FromTheFirstIn

I believe a perfect solution exists. With one big caveat: you need a very clear set of requirements. Every constraint on the table. Tighten those enough and something interesting happens, you end up with only one possible solution.

This is the kind of naive thinking people are actually pushing back on when they’re telling you not to build perfect products. One of the constraints that the author refuses to acknowledge is the social constraint of leading people to a shared goal- if people keep telling you your perfect solution isn’t viable and they do not want to do it, then it’s not perfect, is it?

In general, any approach that believes there’s a One True Answer is either looking at trivial problems or naive egocentrism, in my experience.

If it makes you feel better, most people don’t think they’re real intelligences

The only way to mitigate the damage an LLM can do because of prompt injection is to limit what that LLM can do in the first place. That’s what they mean by limiting its usefulness. If an LLM has access to an api and I want it to abuse that API in some way, I can attack its prompt and eventually get it to use the api the way I want

I don’t think asking if someone lives in SF is ad hominem, but go off

I think they have the same app, but they don’t use it like this. It seems like you’re confident this happens a lot- do you live in San Francisco?

You’re the poster- do you live in the bay?

This is strictly a San Francisco tech problem. This isn’t happening to most people

It’s a bit like saying copy and paste must be intelligent because its output is indistinguishable from intelligent output. Reproduction of intelligence has never been equivalent to intelligence- that’s why we look down on copying, plagiarism or derivative work.

Would you say those humans understand what they’re talking about?

Yes, I think we simply disagree. I think you know what LLMs are based on our other thread and if you think something important happens when you get a large enough context vector I don’t think there’s much I can say to change your mind. It seems unlikely to me!

Way better- thank you!

We’re in all the same threads lately!

I think if your definition of consciousness can fit these things then you’re more open minded than I care to be. Consciousness isn’t really guessing the next thing to say- it’s hard to say what it is, obviously, but blindly feeling forwards with each new conversation doesn’t seem like consciousness or learning to me.

The styling on the website makes me feel like my phone is a cylinder

While being slow to pass judgment or disregard an approach is a valuable trait in a senior Eng, I think 3 years is plenty of time to wait for proof of concept to pan out. It’s not panning out, it doesn’t seem on the verge of panning out, and soon the real cost is going to be passed on and the subsidies will end. LLMs see ripe to be the new IDEs, but not the new Engineers

To acquire new knowledge and build your understanding. They don’t understand so they can’t learn

You’re right, I’m very suspicious of HN when it comes to AI apologetics, but I shoulda trusted the parent commenter more.

Yeah, it not a linear cap (x% entropy doesn’t mean x% wrong) but it does seem like a hard cap. To be honest, the more I’ve understood scaling laws the more I think that the elephant in the LLM room is the entropy of the language. It explains why coding languages are so much more tractable (they’ve got WAY less entropy) and it explains why we haven’t seen a step function in capabilities for LLMs since GPT-4 outside of making specific toolings for particular contexts. I think E is coming to dominate and there isn’t a workaround for it.

This is a great read! I never realized the scale of the effort to find that BSD vulnerability- helps put things in perspective

Isn’t the limit exactly what you’re describing? There’s always uncertainty, and your asymptote can approach its limit but it does have a limit. That’s the limit to the intelligence. And this is just for cross entropy loss- even if you could get loss to 0, I’m still not convinced at all that an enormous semantic map and its convoluted geometries amounts to intelligence.

Agents are always combining the same underlying weights to their inputs, relying on the same maps of semi-semantic space and the relationships between those that it was leaning towards at training time. The fact that it’s successful in making lots of people have an Eliza effect doesn’t make it understand something. It’s simulating understanding based on an enormous corpus of text, much of which is people working through things or sharing an understanding of something. Unless you believe that all intellectual activity is about finding the space between words you shouldn’t believe LLMs have any chance at understanding anything.

The first sentence is “understand or learn any intellectual task that a human can.” Whatever you think of the benefits of LLMs, they don’t understand and they can only learn during the training period and with very minor adjustments in post training. So, no I don’t think any of these models are generally intelligent.

My understanding is that the true entropy floor of a language is intractable- regardless of context length there will be “unpredictable” tokens where cross entropy loss is bound to happen. Even with infinite parameters and data you’ll still have a chance at failing to predict the next token correctly a decent chunk of the time.

Also, linear gains in context length scale quadratically with compute because of attention, so depending on context growth means taking a bath on GPUs for as long as you can, right?

And sitting right next to the data and compute factors in every cross entropy loss equation is the entropy of the language, which is just a fixed constant. There’s such a hard cap on cross entropy loss training and I never hear it come up!

You’d have to really stretch the definition of AGI to make the current models fit

Cheating is always logical for the cheater unless they’re discovered and held to account. I’m not sure what your comment is pointing out besides that it’s possible, but worth saying: just because you can cheat and would benefit from cheating doesn’t mean you’re not culpable for cheating.

Buddy it’s not even going to replace 1%

But they are, so it’s demoralizing to be punished for not hitting the jackpot

Maybe slop?

The deciding power they’re talking about is the power to decide to use or not use an LLM.