robbrown451

I was built with itself, and is essentially optimized for apps like itself.

It started off slow and as the system got better, it sped up its own development, basically exponentially.

Sometimes it got a bit weird, where I would be improving the protocol the LLM uses to save edits, but it would assume the changes that it was actively sending were already in place.

Heh, yeah that's a clever analogy as well. (and thanks!)

" There's no reason to think a super intelligence would be totally fine being a slave to apes."

Sure there is. Intelligence doesn't give us our selfish motivations, natural selection does. We have similar motivations to C elegans, that has all of 302 neurons. Stay alive and have sex.

Honeybees don't though. They are about halfway between humans and C elegans when it comes to cognitive power. But they are not selfish because they don't reproduce directly (I'm talking about the worker bees). So they will sting even though it kills them. All their behavior is consistant with this.

I wouldn't call the harness an AI, but I might call a tool that plays a major role in creating another one like it "recursive self improvement." For instance in the industrial revolution a metal lathe and a milling machine were instrumental in creating the next generation of themselves. Same thing with a robot that is fabricated by similar (i.e. older model of the same) robots. All of them lead to exponential improvement.

Where do you see evidence of vibe coding the harness? (and who are you talking about, Anthropic or the link I shared?)

It seems odd to complain about a AI coding tool being coded with AI. That's just eating your own dog food. In my opinion it makes it better, because the tool is very well tested.

Speed isn't really a big deal for me. I want good quality code. It's already able to generate code 10-100X as fast as I could code it myself.

Anyway, are you speaking of the harness? The harness on mine isn't AI, so speed just isn't an issue.

I'm not sure what I am looking at with chatjimmy.... what is special about it? Speed?

I'm also not sure what you mean by "we aren't there yet." Where?

Sorry, not trying to be difficult or dense, I'm just not sure what you are referring to.

mostly because most of the focus is on exploding the context and parameters.

Large context allows a surprising amount of "learning" to happen at inference time rather than training time. I think that is relatively unexplored. As long as the model itself has passed a certain threshold of smarts, and the context is large enough (Gemini and its million token context being WAY past that point) you are not really limited by the model, you are only limited by how good the stuff you feed into that context is.

That's what happened when, nearly a year ago, I saw a major leap in capabilities that happened entirely on my end.... not in the AI, but in code written by the AI. I found it genuinely frighting to be honest. I think OpenClaw tapped into something similar, which seemed to surprise a lot of people. There were latent capabilities in the AI that were unknown until brought out by a clever harness.

Start off with my video!!! You can also try it with zero setup (you can code right there on the static web page, it will save your edits in the browser indexed DB, and hotpatch them back into the code before it runs it.... also you can grant permission to the browser to read/write to a local directory)

recursi.dev

Seriously, I'm looking for collaborators.

There's upwards of 80,000 lines of code in the editor system, a lot to it to make sure that even newbies don't get stuck.... so that's kind of proof the system works since it doesn't break down when the codebase grows large.

I used to think that, but ended up going the other direction, partly because I don't have the wherewithall to build a model but then I realized, with existing models that can take more than a tiny amount of context, you can just let any model bootstrap itself with a good prompt sent by the system.

There's a ton of other tricks to it, but mostly keeping the protocol simple for the AI so it can concentrate on coding logic and not stuff like managing BS boilerplate, dependencies, etc. (for instance I make extensive use of things like abstract syntax tree library to help with surgical edits from the LLM)

That said, I would be very open to collaborating with someone who builds such small models, I don't think the system strictly needs it, but it also could have some extra power if it had it.

Do code harnesses that build themselves count as recursive self improvement, or does it need to be the AI itself to qualify for the term?

I always was fascinated (obsessed?) by robots that build robots, or even things like this that can contribute a lot to making the next version of itself: https://buildyourcnc.com/products/cnc-machine-blacktoe-v4-2x... (cnc router that cuts plywood, and is made out of cnc-router cut plywood)

This is my own effort at an AI assisted coding environment optimized for building itself: https://recursi.dev/ (just launching it, hope its ok to mention it, it is free/open source.... here is the HN link that has gotten no love yet: https://news.ycombinator.com/item?id=48401022 )

Personally I think harnesses are as important as the AI itself, and have this crazytheory that even if the models stopped improving today we could still have massive advances in the harnesses alone.

Rob Brown here, the developer of this app and ecosystem. I've poured my heart and soul into this project for over a year. I think its crazy powerful and cool and fun. I hope you'll check it out!!!! I'm here for questions.

When in history has being idle not been a problem?

If AI and robots are able to do all the jobs, being idle isn't the negative it has always been.

All through history, you needed lots of non-idle people to do all the work that needed to be done. This is a new situation we are coming upon.

I'm in their time zone, and was just planning to stop with my bad habit of staying up working till 4 am and waking up at noon.

So much for that plan.

For most things they don't need to be "human equivalent." I'd be willing to be the current crop of robots we're seeing could do most tasks like vacuuming, cooking, picking up clutter, folding laundry and putting it aways, making beds, touch up painting, gardening etc. It seems to be getting better very fast. And if mechanical tendons break, you replace them. Big deal. You don't even need a person to do the repair.

I'm having trouble understanding what they want to "upskill" those people to do.

What skills won't be replaced? The only ones I can think of either have a large physical component, or are only doable by a tiny fraction of the current workforce.

As for the ones with a physical component (plumbers being the most cited), the cognitive parts of the job (the "skilled" part of skilled labor) can be replaced while having the person just following directions demonstrated onscreen for them. And of course, the robots aren't far behind, since the main hard part of making a capable robot is the AI part.

It was cited from a Newsweek article, and Cliff said this about it later: "Of my many mistakes, flubs, and howlers, few have been as public as my 1995 howler ... Now, whenever I think I know what's happening, I temper my thoughts: Might be wrong, Cliff ..."

You may be right about humans biasing toward easiest to obtain information, but that doesn't say "don't use AI assistance", it says "use care when using AI assistance".

Also, Cliff wasn't saying the information was easier to use, since in his case, it was actually harder to use than just looking it up in a printed encyclopedia or the like. But none of the problems he mentioned were inherent problems with the internet, they were because it was a brand new medium still working out its kinks. AI may well be harder to use for coding right now, at least for many use cases. However, a look at the bigger picture strongly suggests it is the future, just as a look at the bigger picture in 1995 would have suggested that the internet was the future, at least for answering questions like "when was the battle of Trafalgar?"

This is consistent with my horse/car analogy: the car wasn't the problem, the problem was people who assumed cars were going to keep themselves on the road like a horse would naturally do. You can get a huge gain, but you have to be smart about how you use it.

I'd suggest enjoying that vindication while it lasts.

From my perspective, your perspective is like a horse and buggy driver feeling vindicated when a "horseless carriage" driver accidentally drives one into a tree. The cars will get easier to drive and safer in crashes, and the drivers will learn to pay attention in certain ways they previously didn't have to.

Will there still be occasional problems? Sure, but that doesn't mean that tying your career to horses would have been a wise move. Same here.

(Also, this article is about "poisoned ChatGPT-like tools." Which says very little about using the tools that most developers are using)

I'm always reminded of this: "Logged onto the World Wide Web, I hunt for the date of the Battle of Trafalgar. Hundreds of files show up, and it takes 15 minutes to unravel them—one's a biography written by an eighth grader, the second is a computer game that doesn't work and the third is an image of a London monument. None answers my question, and my search is periodically interrupted by messages like, "Too many connections, try again later."" -- Cliff Stoll, 1995

Imagine if a regular for profit startup did that. It gets 60 million in initial funding, and later their valuation goes up to 100 billion. Of course they can't just give the 60 million back.

This is different and has a lot of complications that are basically things we've never seen before, but still, just giving the 60 million back doesn't make any sense at all. They would've never achieved what they've achieved without his 60 million.

I don't see how opening it makes it safer. It's very different from security things, where some "white hat" can find a security, and they can then fix it so instances don't get hacked. Sure, a bad person could run the software without fixing the bug, but that isn't going to harm anyone but themselves.

That isn't the case here. If some well meaning person discovers a way that you can create a pandemic causing superbug, they can't just "fix" the AI to make that impossible. Not if it is open source. Very different thing.

There are differences but there are also similarities. I think the similarities are more important, both when you're driving and interacting online, you have conflicting agendas, which could be a simple as when driving you're trying to get there as soon as possible, and when you are using an online message board you're either trying to get your point accepted or you trying to make yourself look good and smart.

The point, though, is that if you're not gonna have to interact with these people in the future, and there are otherwise no repercussions to being nasty, you're more likely to be nasty.

Any digital piano functions as a midi controller, and many of them have weighted keys. And there are a few "pure" midi controllers that have 88 weighted keys, such as the M-Audio Hammer 88 or StudioLogic SL88.

"This is not a representation of the real world, of facts, but simply a product of its training."

Tell me how that doesn't apply to the human brain as well.

"LLMs do not directly model the world; they train on and model what people write about the world"

This is true. But human brains don't directly model the world either, they form an internal model based on what comes in through their senses. Humans have the advantage of being more "multi-modal," but that doesn't mean that they get more information or better information.

Much of my "modeling of the world" comes from the fact that I've read a lot of text. But of course I haven't read even a tiny fraction of what GPT4 has.

That said, LLMs can already train on images, as GPT4-V does. And the image generators as well do this, it's just a matter of time before the two are fully integrated. Later we'll see a lot more training on video and sound, and it all being integrated into a single model.

It certainly doesn't "look up" text data it has seen before. That shows a fundamental misunderstanding of how this stuff works. That's exactly why I use the example above of Alpha Zero and how it learns to play Go, since that demonstrates very clearly that it's not just looking things up.

And I have no idea what you mean by saying that it has no concept of true or false. Even the simplest computer programs have a concept of true or false, that's kind of the simplest data type, a boolean. Large language models have a much more sophisticated concept of true and false that has a lot more nuance. That's really a pretty ridiculous thing to say.

I agree with Hinton, although a lot hinges on your definition of "understand."

I think to best wrap your head around this stuff, you should look to the commonalities of LLM's, image, generators, and even things like Alpha Zero and how it learned to play Go.

Alpha Zero is kind of the extreme in terms of not imitating anything that humans have done. It learns to play the game simply by playing itself -- and what they found is that there isn't really a limit to how good it can get. There may be some theoretical limit of a "perfect" Go player, or maybe not, but it will continue to converge towards perfection by continuing to train. And it can go far beyond what the best human Go player can ever do. Even though very smart humans have spent their lifetimes deeply studying the game, and Alpha Zero had to learn everything from scratch.

One other thing to take into consideration, is that to play the game of Go you can't just think of the next move. You have to think far forward in the game -- even though technically all it's doing is picking the next move, it is doing so using a model that has obviously looked forward more than just one move. And that model is obviously very sophisticated, and if you are going to say that it doesn't understand the game of Go, I would argue that you have a very, oddly restricted definition of the word, understand, and one that isn't particularly useful.

Likewise, with large language models, while on the surface, they may be just predicting the next word one after another, to do so effectively they have to be planning ahead. As Hinton says, there is no real limit to how sophisticated they can get. When training, it is never going to be 100% accurate in predicting text it hasn't trained on, but it can continue to get closer and closer to 100% the more it trains. And the closer it gets, the more sophisticated model it needs. In the sense that Alpha Zero needs to "understand" the game of Go to play effectively, the large language model needs to understand "the world" to get better at predicting.

Well fully sentient doesn't mean it is superintelligent.

Exactly what are you doing here then?

But hey I guess I can do this too. How's this? Using cringe as an adjective is cringe.

You're saying the study has no grounding in how brains work? I'd think a more reasonable conclusion would be that the neuroscientists involved have no grounding in how artificial neural networks work.

It seems the whole point is to bring in additional details of how brains work, that the think may be relevant to artificial NNs.

I dunno. My comment complained about the parent comment not adding positively to the discussion. And gave at least a bit of support for that complaint.

Would you have preferred I emulate your style, and complain while providing no support for my complaint?

Ok.

I can't agree with the dismissiveness of this comment, and frankly I find its tone out of line and not with the spirit of Hacker News.

There are insights that can come from studying the brain, that do indeed apply. Some researchers may not glean anything from such studies, and some may. I have no doubt that as neural networks get more an more powerful, we will continue to find more ways they are similar to the brain, and apply things we've learned about the brain to them.

I certainly prefer to see people making comparisons of neural networks to the brain, that the old "it's just a glorified autocomplete" and the like.

Relax.