krupan

Hugging Face: Some super smart AI agent hacked us

OpenAI: That was us. It was our AI that was smart enough to do this. We even tried to stop it (you know, after we started it), but it outsmarted us. Man, our AI really is super smart. You can pay us to use it, by the way.

This is either:

- massive skill in one area (making a smart AI) and massive incompetence in another (creating safe test environments)

- harmlessly hack on purpose in order to do some clever marketing

- Maliciously hack a competitor on purpose, bungle the hack, own up to it but call it an accident, all while subtlety touting your product

More oversight hinders anyone that is trying to catch up to OpenAI. It's something OpenAI wants

Fear is exactly what OpenAI and Anthropic are hoping for. Don't let drive you

I mean, once a government starts literally printing money to solve their problems are they not then thoroughly corrupt and dictatorial? You kinda qualified your question into pointlessness ;)

Also, the exact process by which governments create money out of nothing (directly printing it, issuing bonds, changing the materials used for coins, changing numbers in a database, etc.) doesn't really matter, does it?

The term "user" maybe made sense before any software was commercialized. It differentiated the programmer of the software from the non-programmer that used the software. Even then it was sometimes used derogatorily (especially when people were sneered at as "Lusers"). In the original Tron the Good Guys had to fight for the users.

Customer is a much nicer term. Words do matter.

Possibly the author didn't dig this deep into all AI Agent Harnesses, just Open Code?

I think you are correct on how it's playing out because AI cannot write software very well all on its own. The dream is that AI is so good that it can write all the code you need from scratch, replacing all and any code written by anyone else, but that's not happening

"One participant at the retreat noted that permissive licensing was a profound collective mistake, serving as a legal mechanism that enabled the world’s largest corporations to cannibalize volunteer labor..."

Agreed. Linux and GNU did and still do so well because of the GPL. Red Hat built a billion dollar business on GPL software. Tons of Linux developers are payed great salaries by competing corporations that otherwise collaborate on Linux, because none of them are allowed by the GPL to make proprietary changes to the code

You still have people with very tempting access to a whole lot of data. I don't see how this is different from the current situation other than it's just different (corruptible) people with access.

I agree with you 100% that cars are super dangerous, but that's a whole 'nother discussion

C'mon, you aren't going far enough. Catching someone after the crime was committed is not preventing the crime. You need AI to watch the cameras non stop and alert you before someone commits a crime ;-)

The structure that prevents abuse is, don't do mass surveillance

How? Whatever the system is now is clearly not preserving privacy

You make it sound so easy. Have you ever worked in QA?

You just reminded of the term Nuclear Winter. Is that still a thing?

How does the global temperature relate to El Nino temperature? Do we know without having El Nino data?

We don't, sadly

And we should have started at least 30 years ago. But better late than never?

Everyone's a prophet now

I see it. I'm with you. It does feel lonely though, doesn't it?

There's a difference between criticizing and saying, "I'm sorry, I don't understand what you are trying to say."

This argument is so old and tired. It started long before LLMs. People thought senior engineers could write a detailed spec and then the actual coding could be outsourced to just any bunch of cheap programmers. It never worked. A detailed enough English specification doesn't look very different from code written in our current high level programming languages

The end is where you said people should still learn to program, I wouldn't know that you said that without reading to the end.

I think the gist of what the author is saying is, AI is not good enough to just give one vague prompt to and let it go, but it's good enough for you to give it a "design" and then not worry about the actual code it writes. But you need to do a lot of QA still. And you still need to learn to code and write some code, I guess so you can give the LLM good instructions? But if giving good enough instructions requires some level of coding skill, how are you going to gain that skill if you don't do much programming or reading code?

It all sorta feels like an old guy (he says he's old in TFA) who forgot how he got to where he is today trying to give advice. Be careful what you believe, young programmers.

I didn't want to say that out loud, but you are right and it's unfortunately hard to understand some of what he's trying to say

If you have not seen Hunt for the Wilderpeople (staring Sam Neil) find it and watch it today!

No, because nobody has been coding the same way since 1999. But that has nothing to do with LLMs

I love how many people are tech prophets now

It's a big precisely because so many people are blindly trusting AI company marketing. It blows my mind how many people, especially here on once-subversive hacker news, are not just swallowing the marketing but they are doing the marketing for these companies for free. That's why it needs to be talked about!

Memory safety is a source of bugs in languages without garbage collection and/or whatever rust does, however memory safety is not THE source of bugs. If it were then we'd expect code written in JavaScript or Python (memory safe languages) to have zero bugs, and that's definitely NOT the case. When you get the hang of C and C++ memory bugs are not very common compared the normal bugs that all software has. They still can happen but people have gotten the idea that every C program is riddled with them without seeming to realize that code written in C is the foundation for almost everything and has been for a long time. Compilers, operating systems, device drivers, network stacks, core utilities, even games are all written in C or C++ and we are not drowning in killer memory related bugs.