resident423

You mean for AI consciousness or existential risk? I think Anthropic downplay the existential risk (they might talk about it now and then, but they still build frontier models) but are overly confident about AI being conscious, and I think these two things are pretty strongly related to each other.

I agree with this, but I don't think Anthropic would, they appear to be much more convinced than what I think would be reasonable.

It seems like a simple solution but if Anthropic was actually to dedicate all its resources this way wouldn't the investors just demand a new CEO?

I think Anthropic believe these risks, but I also think they've spent so much time talking to Claude that they've pretty much lost their minds now. Anthropic have a model welfare department and have numerous times suggested that Claude is conscious and has human like emotions.

My smoke detector has gone off three times now, where is the fire?

I suppose you end up hating it either way, if you're a better developer than it you don't like it because it sucks, and if it's a better developer than you then you just feel obsolete

You mean that self improving AI is very far away? I'm not sure how much I believe that now it's solving Erdos problems?

but can't come up with revolutionary theories that were never in the training set.

Can you elaborate? I don't think the solution to the unit distance problem was in the training set, but I'm guessing you mean there's some higher bar for revolutionary theories LLMs cant reach? If so where do you expect the limit will be?

I agree with all of these points, my view is just that open source doesn't really do much to prevent it. I also think it adds the additional danger of making dangerous capabilities widely available to anyone, like the ability to design novel viruses which is something that we can't really defend against once it's out there. If anything, putting this kind of capability in the hands of anyone with a GPU could create justification for a mass surveillance state or further concentration of power.

I also just don't think the open source movement has much chance of competing with the city sized data centres owned by Anthropic and OpenAI, or the hundreds of billions of dollars they have available to hire the best researchers. It costs hundreds of millions to train a frontier model, this kind of compute isn't available to the open source community.

I can understand this for hacking tools, but I'm not really sure how we fix the security issues on the CBRN side? We can't patch the human body like we can with software, so if the model has strong biological capabilities and is released open source, what stops it being used to construct new viruses and things like this?

I would say I agree with Anthropic on open source for the reasons stated above like cyber crime, CBRN etc, but I'm interested to hear the other side of the argument. What would be the argument for open source over closed source?

At the moment is the significant part, how long do we expect this to last now that AI is capable of generating novel ideas like solutions to Erdos conjectures?

CONFLICTS OF INTEREST

DC has conducted industry-sponsored research involving creatine supplementation and received creatine donations for scientific studies and travel support and speaking honoraria for presentations involving creatine supplementation at scientific conferences and on social media. In addition, DC serves on the Scientific Advisory Board for Alzchem and Create (companies that manufacture creatine products) and as an expert witness/consultant in legal cases involving creatine supplementation. NF declares no conflicts of interest

A nice unbiased study, I may as well ask Exon Mobil for their opinion on climate change at this point

I don't think that's true, from what I've seen the people on here are pretty consistently anti surveillance

All of these discussions seem to assume there's a limit at human level intelligence beyond which AI can progress no further. What stops the AI companies taking their human level model and training it even more until it's superhuman?

Pdoom would be the most important for me, everything else depends on us being able to control the AI.

But beyond that there's still problems like concentration of power and surveillance, permanent loss of jobs, cyber and bio security. I'm not convinced things will go well even if we can avoid these problems though. I try to think about what the world will be like if AI becomes more creative than us, what happens if it can produce the best song or movie ever made with a prompt, do people get lost in AI addiction? We sort of see that with social media already, and it's only optimizing the content delivery, what happens when algorithms can optimize the content itself?

There's isn't even a solution for how to control highly capable systems at all, everyone wants to decide what to do with the AI before they've even solved the problem of controlling it.

It's like how everybody imagines their lives will be great once they're a millionare, but they have no plan for how to get there. It's too easy to get lost dreaming of solutions instead of actually solving the important problems.

130000 so far https://jobloss.ai/

I feel like these articles are just a reasurance for people who don't want to accept that AI will automate their jobs. It becomes easier to focus on a lesser group of AI users and feel superior than to confront the reality of things.

I don't really like the idea of an app telling me how to manage my friendships, my view is that people can handle their relationships without intervention. I'm not sure what problem it is trying to solve.

No, but I'm interested to know what it is?

I haven't used stable diffusion enough to have a strong opinion on it. But my thinking is LLMs have only recently started contributing novel solutions to problems, so maybe there is some threshold above which there's less sloppy remixing of training data and more ability to form novel insights, and image generators haven't crossed this line yet.

Remember when people thought solving Erdos problems required intelligence? Is there anything an LLM could ever do that would cound as intelligence? Surely the trend has to break at some point, if so what would be the thing that crosses the line to into real intelligence?

Solving open math problems is strong evidence of intelligence so there's not really any need for rationalization? I don't understand why intelligence would require intent or motive? Isn't intent just the behaviour of making a specific thing happen rather than other things?

There's probably a limit to how intelligent something can be with no long term memory, but solving Erdos problems in 80 minutes is clearly not above it, and I think the true limit is probably much higher than that.

What your describing sounds more like the model is lacking awareness than lacking intelligence? Why does it need to know it solved the problem to be intelligent?

I wonder if the rationalizations people come up with for why this isn't real intelligence will be as creative as ChatGPTs solution.

Can both not be true?

Why hopefully?

There are also large organizations at Meta focussed on the optimal distribution of scam ads to the elderly.

https://www.reuters.com/investigations/meta-is-earning-fortu...

Meta employees are not typically known for their deep concerns about privacy.