Many AI safety orgs have tried to criminalize currently-existing open-source AI

https://1a3orn.com/sub/machine-learning-bans.html

by sroussey • 3 years ago

385 373 3 years ago

roenxi2• 3 years ago

There are many types of safety. For example, protecting your profits! I'd imagine that if we trace the money back these organisations will look a lot like lobbyists for existing companies in the AI space. I recall Microsoft's licence enforcement effort was done with that sort of scheme, I think they used the BSA [0]. It has been a while though so maybe it was a different group.

Anyway, point being, if they can lobby for something unpopular under a different brand, that is how to do it. Much less PR risk.

[0] https://en.wikipedia.org/wiki/Software_Alliance

Xelynega7• 3 years ago

I have to believe this is the case.

The vast majority of "ai safety" news I see is about some dumb shit someone is getting called out for supporting and the responses are always "ai safety is such a joke since they focus on these issues instead of real issues".

Who does this benefit to have the "popular ai safety" news/people be ridiculous? It's not furthering the issues people seem to care about, but it does give a bunch of ammunition for Microsoft/openai/etc. fans to use to shutdown any discussion of AI safety issues.

65a• 3 years ago

Never underestimate the desire of existing systems of control or power to self-propagate.

rmbyrro2• 3 years ago

You mean existing systems of lobbyists representing control-hungry people who want AI power only to themselves, right?

Yeah, as the post shows, we're well aware of these systems.

google234123• 3 years ago

I’m curious if the board of Anthropic AI will eventually try and kill the company because if I’m not mistaken many of the members share this mindset

sillysaurusx• 3 years ago

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

I wish I could be more eloquent on this point, but I’ve mostly just been depressed about this seeming inevitability.

Hopefully it won’t be the case. But how could it be otherwise? Hundreds of thousands of people are mad at openai and midjourney for doing exactly what open source AI needs to do in order to survive: fine tune or train from scratch.

As soon as some politician makes it a platform issue, it seems like the law will simply be rewritten to prevent companies from using training data at will. It’s such a compelling story: "big companies are stealing data owned by small businesses and individuals." So even if the court cases are decided in OpenAI’s favor, it’s not at all clear that the issue will be settled.

idle_zealot• 3 years ago

The advantage that open AI (not the company) has is that if using copyrighted content as training data without licensing it is found to be illegal, they can just keep doing it. There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients) that big tech cannot compete with lest they face legal consequences. Basically what I'm saying is that the open source community will continue to use books3 and scraped images to train while facebook and the like get stuck in legal quagmire.

Of course, this ignores the fact that popular "open" models of today were actually trained with facebook or other companies' computational resources, so unless some cheaper way to train models were developed we would actually be stuck with proprietary models trained with lots of compute but unable to use unlicensed training data, and open models that can use whatever data they like but must operate in the shadows without access to much compute for training.

livueta• 3 years ago

You're right that there are areas like torrenting where big tech can't really go for fear of legal consequences, but there's also the cat-and-mouse game played by IP holders against, say, torrent trackers, which leads me to think that

open models that can use whatever data they like but must operate in the shadows without access to much compute for training

sounds pretty analogous to private trackers, where high-quality stuff is available but not in full public view. If rightsholders and big tech, abetted by states, crack down on open models I think you're right that the open source community will continue to train on liberated content, but it's not going to be as open and free-flowing as things are now. Going back to the compute problem, I can imagine analogues of private trackers where contributors, not wanting to expose themselves to whatever the law-firm letter/ISP strike analogue will be in this space, use invite-only closed networks to pool compute resources for training on encumbered content.

ayende• 3 years ago

As long as the cost for training a model is in the 7+ figures, that means that any such open model is bound to be tracked to someone with deep enough pockets to sue.

Consider that you just spend a few millions on training a model on copyrighted data. Release it would reveal that, problem.

I _guess_ you can try doing training in the public, like Seti @ Home or something like that, which distributes the risk? But no idea if this is even possible in this context.

shkkmo2• 3 years ago

There's plenty of FOSS software basically designed to violate copyright law (comic readers, home media center servers/clients, torrent clients)

The key difference is that those projects don't violate copyright themselves, but facilitate users doing so. If training without a license is infringement then projects that are doing so will struggle to host code/models publicly or access other parts of internet infrastructure.

grotorea• 3 years ago

That may be partly true but are small open source groups going to have the financing to do AI training that can compete with the big techs? I think the most likely scenario is for some other country with a less strict view of copyright (ahem) to continue developing AI without caring for the input data.

mrtksn• 3 years ago

We have infinite data, a microphone and a camera can generate huge amount of it and the public domain literature is wast. Billions of people learn like that everyday.

zarzavat• 3 years ago

It’s impossible to learn any technical topic from 70+ year old books. The public domain is small and basically zero if you want to learn anything current. A microphone and camera is fine for learning about daily life, but you cannot get “book smarts” without copyrighted media.

PeterisP2• 3 years ago

The public domain literature is big, but not that big and very far from infinite data - all the major current models already include all of the public domain literature that has ever been digitized and much more (so all the public domain literature not only isn't abundant, but it isn't even sufficient to make anything competitive with even last generation models), and we would really like to get 10x or 100x more data than all the literature ever published (both PD and not), if we could. And we probably can, since the vast, vast majority of what people write happens in other contexts and only a tiny fraction of written words end up as published literature.

torginus2• 3 years ago

my question is - if it's deemed illegal to train AI using proprietary data sets, what's to stop companies from using their already existing LLMs to generate training data?

kolinko4• 3 years ago

The fear is that you won’t get better model from training on synthetic data from a worse model, and you will miss a lot of modern day knowledge.

Imho the first is not necessarily so, but the second will be a real hindrance. You can talk about the pandemic with LLMs because they were trained on people’s comments on the subject, but if we had training limits you wouldn’t be able to do so.

Otoh with many cases a sufficiently smart AI can just reach out to the open source material - e.g. if you want to know about modern day politics of Europe, you can read reports of commissions, and if you want info on a new framework/tech, you can just read source code or scientific papers

FergusArgyll• 3 years ago

There was a famous-ish study about this a couple months ago that showed that llms training on llm generated data get dumber. I can't remember the name.

I'm not sure if that result is holding up though, I've seen some ideas of generating 'textbook quality' data that might work

kolinko4• 3 years ago

Depends on what you want to use AI for - for a smarter AI we could probably do with the open source material, but better training techniques and some innovative breakthroughs in the model.

Also, an interesting side note is that certain countries may actually want their language corpus involved in training AI - so they may have copyrights that are more AI friendly. I imagine Poland or Czech Republic wanting as much as their data to be involved in training LLMs because it gives their cultures more exposure. Even more so with African countries.

gfodor• 3 years ago

The US maintaining its lead is a national security issue. Larry Summers is now on the board of OpenAI. The copyright holders are not going to win, or they are, the training will continue anyway, but the technology will be (for now) kept in the hands of the military and IC.

DiscourseFan• 3 years ago

"big companies are stealing data owned by small businesses and individuals."

I mean, they are.

Unfortunately, AI at this point in time (LLMs, Midjourney, etc.) do appear to be not much more than highly technical and complex forms of intellectual property theft, which are given the futuristic name of "AI" to cover what they are actually doing. That isn't to say that they are entirely performing intellectual property theft; the models themselves are very fascinating in how they function, especially as sort of "extra-rational" entities that clearly have a logic, though not one we can fully understand.

But the sort of philosophical, metaphysical, technological side of LLMs and Midjourney are clearly not being explored by OpenAI in a responsible manner, otherwise they wouldn't have just straight up stolen, seemingly, NYT articles. It's just another example of Silicon Valley VCs using a wonderful technology solely for exploitative profits, just like how google search became a surveillance tool (and plenty of other examples to go along with that).

Its only right that we acknowledge the work of the human beings (all the art, writing, etc.) that went into the creation of the model, instead of pretending that there is some totally non-human machine intellect that is going to take over the world through its vast intelligence and negate all human action and all human work etc. etc. That is just a fantasy generated by wealthy Silicon Valley VCs as justification to themselves and those working under them, and those they are stealing from, to cover precisely that work that they would rather avoid paying for.

3abiton• 3 years ago

Funny thing is that most of the data is provided by us users one way or another "for free", and yet we got no say on how it's used.

ChatGTP• 3 years ago

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

Absolutely not, I truly believe that many, many people will altruistically donate material to an open source foundation where all humans benefit from the models.

People are pissed because Open AI are using their copyrighted material to, make claims about ending humanity, profit hardcore, and keeping their work under their own lock and key.

neurostimulant• 3 years ago

There is a bigger reason why the end of open source AI might be close: as soon as training data becomes licensed, that’s it for open source AI. Poof.

In the West, maybe. I doubt it'll slow down open source AI development pace in places like China.

Buttons840• 3 years ago

Instead of a kid in high school selling the answers to the test, the kid will be selling the weights to a China-trained LLM that can write all your essays.

eurekin• 3 years ago

In case of programming, there could be some mileage with synthetically generated code examples.

whiplash451• 3 years ago

That ship sailed a long time ago.

Training on publicly available data is the new norm.

Governments know that if they forbid it in their jurisdiction, they will fall behind the govs that don’t.

oceanplexian• 3 years ago

Licensed in what country? Some of the best open source models are now coming out of China, UAE, etc. What is the government planning to do with that exactly, ban files on the internet?

csomar• 3 years ago

This will create a rift in the US tech sector. Look at how this played out with Russian sanctions; and this will lead to a similar path for tech and AI. Alternative players will train/open source models to their own advantage and the hardware will find its way to these markets.

The people making the decisions these days are too drunk.

RishabhKharyal• 3 years ago

1. Licensing of data will be huge bottleneck 2. Uncensored results will be used against opensource models questions hovering in dark or grey area 3. Limited compute compared to big corp and model size gap 7B for opensource and closed source would be magnitude bigger

lannisterstark4• 3 years ago

Licensing of data will be huge bottleneck

Will it? At some point it'll be equally (or close to) cheaper as me just building my own *arr stack instead of subscribing to 5 different $20 streaming sites.

Uncensored results will be used against opensource models questions hovering in dark or grey area

Will they care though? You can use whatever you want against me for pirating x, I'm still doing it.

----

Limited compute

THAT. That is the point we want to make. For FOSS projects, compute is the single biggest hurdle. Everything else we can make do.

kmeisthax2• 3 years ago

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned, ala Dune. There are AI harms that don't care about whether or not the weights are available, and are playing out today.

I'm talking about the ability of any AI system to obfuscate plagiarism[0] and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it.

[0] No, I don't mean the training sets - though AI systems seem to be suspiciously really good at remembering them, too.

Llamamoe2• 3 years ago

The by far biggest harm to society of AI is the devalustion of human creative output and replacing real humans with cheap AI solutions funneling wealth to the rich.

Compared to that, an open source LLM telling a curious teenager how to make gunpowder is... laughable.

This entire debacle is an example of disgusting "think of the children!" doublespeak, officially about safety, but really about locking shit down under corporate control.

kolinko4• 3 years ago

It sounds similar to the fears media rang about tat you can find a bomb making tutorial on the Internet - people were genuinely afraid of that.

(Otoh an agi can bring unforseen consequences for humanity - and that’s a genuine fear)

badgersnake• 3 years ago

The biggest harm is the torrent of AI generated misinformation used to manipulate people. Unfortunately it’s pretty hard to come up with a solution for that.

kragen2• 3 years ago

if the 'devalustion' of human creative output and replacing real humans with cheap ai solutions funneling wealth to the rich is in fact the 'by far biggest harm' then basically there's nothing to worry about. no government would ban or even restrict ai on those grounds

even the 'terrists can figure out how to build an a-bomb' problem is relatively inconsequential

what ai safety people are worried about, by contrast, is that on april 22 of next year, at 7:07:33 utc, every person in the world will keel over dead at once, because the ai doesn't need them and they pose a risk to its objectives. or worse things than that

i don't think that's going to happen, but that's what they're concerned about

visarga12• 3 years ago

Why is paraphrasing or using ideas from a text such a risk? If all that was protecting those copyrighted works was the difficulty to reword, then it's already pretty easy to circumvent even without AI. Usually good ideas spread wide and are reworded in countless ways.

JoshTriplett5• 3 years ago

AI safety people are hypocrites. If they practiced what they preached, they'd be calling for all AI to be banned

They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all.

There are risks that apply only to open. There are risks that apply only to closed. But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage.

This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it.

Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk.

If you want to argue, concretely and with evidence, why you think it isn't an existential risk, that's an argument you could reasonably make. But don't portray people as ineffectively doing the thing you think they should be doing, when they are in fact not trying to do that, and only trying to do something they deem more important.

mirekrusin3• 3 years ago

Let's also ban cryptography because nuclear devices/children.

GaggiX• 3 years ago

They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all.

That's not true if you read the article.

kolektiv7• 3 years ago

I'm not convinced the onus should be on one side to prove why something isn't an existential risk. We don't start with an assumption that something is world-ending about anything else; we generally need to see a plausibly worked-through example of how the world ends, using technology we can all broadly agree exists/will shortly exist.

If we're talking about nuclear weapons, for example, the tech is clear, the pattern of human behaviour is clear: they could cause immense, species-level damage. There's really little to argue about. With AI, there still seems to be a lot of hand-waving between where we are now and "AGI". What we have now is in many ways impressive, but the onus is still on the claimant to show that it's going to turn into something much more dangerous through some known progression. At the moment there is a very big, underpants gnomes-style "?" gap before we get to AGI/profit, and if people are basing this on currently secret tech, then they're going to have to reveal it if they want people to think they're doing something other than creating a legislative moat.

rokkitmensch2• 3 years ago

I oppose regulating what calculations humans may perform in the strongest possible terms.

trevyn4• 3 years ago

nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage.

I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

suslik• 3 years ago

What is the reason to believe that LLMs are an evolutionary step towards AGI at all? In my mind there is a rather large leap from estimating a conditional probability of a next token over some space to a conscious entity with its own goals and purpose. Should we ban a linear regression while we're at it?

It would be great to see some evidence that this risk is real. All I've witnessed so far was scaremongering posts from apparatchicks of all shapes and colors, many of whom have either a vested interest in restricting AI research by others (but not by them, because they are safe and responsible and harmless), or established a lucrative paper-pushing, shoulder-rubbing career around 'AI safety' - and thus are strongly incentivised to double down on that.

A security org in a large company will keep tightening the screws until everything halts; a transport security agency, given free reigh, would strip everyone naked and administer a couple of profilactic kicks for a good measure - and so on. That's just the nature of it - organisations do what they do to maintain themselves. It is critical to keep these things on a leash. Similarly, an AI Safety org must proseletyse excistential risks of AI - because a lack of evidence of such is an existential risk for themselves.

A real risk, which we do have evidence for, is that LLMs might disrupt knowledge-based economy and threaten many key professions - but how is this conceptually different from any technological revolution? Perhaps in a hundred years lawyers, radiologists, and, indeed, software developers, will find themselves in the bin of history - together with flint chippers, chariot benders, drakkar berserkers and so forth. That'd be great if we planned for that - and I don't feel like we do enough. Instead, the focus is on AGIs and that some poor 13-year-old soul might occasionally read the word 'nipple'.

lannisterstark4• 3 years ago

But nobody should be developing AGI

Pass. People should be developing whatever the hell they want unless given a good, concrete reason to not do so. So far everything I've seen is vague handwaving. "Oh no it's gonna kill us all like in ze movies" isn't good enough.

whywhywhywhy3• 3 years ago

They are calling for all AI (above a certain capability level) to be banned. Not just open, not just closed, all.

Nah a lot are complaining about the licensing of content because they think it will destroy it but instead would essentially mean image gen ai would only be feasible for companies like Google, Disney, Adobe to build.

Not sure if you could even feasibly make GPT4 level models without a multi year timeline to sort out every licensing deal, by the end of it the subscription fee might only be viable for huge corps.

simiones12• 3 years ago

Because AI safety people are not the strawmen you are hypothesizing. They're arguing against taking existential risks.

The AI safety strawman is "existential risk from magical super-God AI". That is what the unserious "AI safety" grifters or sci-fi enthusiasts are discussing.

The real AI safety risks are the ones that actually exist today: training-set biases extended to decision-making biases, deeply personalized propaganda, plagiarism white-washing, creative works bankrupting, power imbalances from control of working AI tech etc.

EVa5I7bHFq9mnYK• 3 years ago

> But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage.

Now please fly to North Korea and tell Mr. Kim Jong Um what he should or shouldn't be doing.

DebtDeflation• 3 years ago

They're arguing against taking existential risks. AI being a laundering operation for copyright violations is certainly a problem. It's not an existential risk.

Give an example of an "existential risk". An AI somehow getting out of control and acting with agency to exterminate humanity? An AI getting advanced enough to automate the work of the majority of the population and cause unprecedented mass unemployment? What exactly are we talking about here?

I'm actually a lot more concerned about REAL risks like copyright uncertainty, like automating important decisions such as mortgage approvals and job hires without a human in the loop, and like the enshittening of the Internet with fake AI-generated content than I am about sci-fi fantasy scenarios.

brigadier132• 3 years ago

But nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage.
Because AI safety people are not the strawmen you are hypothesizing

You yourself are literally the living strawman.

If you want to argue, concretely and with evidence, why you think it isn't an existential risk

No, you are the one advocating for draconian laws and bans. It is your responsibility to prove the potential danger.

PakG1• 3 years ago

Kind of makes me wish there was a nonprofit organization focused on making AI safe instead of pushing the envelope. Wait, I think there was one out there....

xyzzy123• 3 years ago

Right now it's not even economic to prove that non-trivial software projects are "safe" let alone AI. AI seems much worse, in that it's not even clear to me that we can robustly define what safety or alignment mean in all scenarios, let alone guarantee that property.

nradov2• 3 years ago

what a silly statement. There is no way to robustly prove "alignment". Alignment to what? Nor is there any evidence that any of the AI work currently underway will ever lead to a real AGI. Or that an AGI, if developed, would present an existential risk. Just a lot of sci-fi hand waving.

variadix• 3 years ago

I don’t think any of the proposals by AI x-risk people are actionable or even desirable. Comparing AI to nukes is a bad analogy for a few reasons. Nukes are pretty useless unless you’re a nation state wanting to deter other nations from invading you. AI on the other hand has theoretically infinite potential benefits for whatever your goals are (as a corporation, an individual, a nation, etc.) which incentives basically any individual or organization to develop it. Nukes also require difficult to obtain raw materials, advanced material processing, and all the other industrial and technological requirements, whereas AI requires compute, which exists in abundance and which seems to improve near exponentially year over year, which means individuals can develop AI systems without involving hundreds of people or leaving massive evidence of what they’re doing (maybe not currently due to computing requirements but this will likely change with increasing compute power and improved architectures and training methods). Testing AI systems also doesn’t result in globally measurable signals like nukes. Realistically making AI illegal and actually being able to enforce that would require upending personal computing and the internet, halting semiconductor development, and large scale military intervention to try to prevent non-compliant countries from attempting to develop their own AI infrastructure and systems. I don’t think it’s realistic to try to control the information on how to build AI, that is far more ephemeral than what it takes to make advanced computing devices. This is all for a hypothetical risk of AI doom, when it’s possible this technology could also end scarcity or have potentially infinite upside (as well as infinite down side, not discounting that, but you have to weigh the hypothetical risks/benefit in addition to the obvious consequences of all the measures required to prevent AI development for said hypothetical risk). I’ve watched several interviews with Yudkowsky and read some of his writing, and while I think he makes good points on why we should be concerned about unaligned AI systems, he doesn’t give any realistic solutions to the problem and it comes off more as fear mongering than anything. His suggestion of military enforced global prevention of AI development is as likely to work as solving the alignment problem on the first try (which he seems to have little hope for).

EDIT: Also, I’m not even sure that solving the alignment problem would solve the issue of AI doom, it would only ensure that the kind of doom we receive is directed by a human. I can’t imagine that giving (potentially) god-like powers to any individual or organization would not eventually result in abuse and horrible consequences even if we were able to make use of its benefits momentarily.

Simon_ORourke4• 3 years ago

You forgot to add "rent-seeking hypocrites". Not one of them actually advance either social concerns or technical approaches to AI. They seem to exist in a space with solicits funding to pay some mouth-piece top-dollar to produce a report haranguing existing AI model for some nebulous future threat. Same with the clowns in the Distributed AI Research Institute, all "won't somebody think of the children" style shrieking to get in the news while keeping their hand out for funding - hypocrites is right!

kridsdale16• 3 years ago

They’re a clergy demanding tithes to keep writing about divine judgement.

torginus2• 3 years ago

I wonder how long would it take this to get fixed, if I fed some current best-seller novels into an LLM and instructed it to reword it, renaming the characters and places, and shared the result publicly for free?

Although I fear the response would be that powerful AI orgs would put copyright filters in place and lobby for legalization for mandatory AI-DRM in open source AI as well.

ben_w17• 3 years ago

What you're describing sounds like a search and replace could already do it.

If you mean something more transformative, did Yudkowsky get a licence for HPMOR? Did Pratchett (he might well have) get a licence from Niven to retell Ringworld as Strata? I don't know how true it is, but it's widely repeated that 50 Shades of Grey was originally a fan fiction of the Twilight series.

Etc., but note that I'm not saying "no".

mtillman2• 3 years ago

Important to remember that in Dune, the AI made the right decision which precipitated a whole lot of fun to read nonsense.

kridsdale16• 3 years ago

I’ve only read 2 novels and want to get in to this. What title(s) cover the machine war?

nerdponx• 3 years ago

It's way too late to ban any of this. How do you propose to make that work? That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it. And where do you draw the line? Is my XGBoost model "AI", or are we only banning generative AI? Is a Markov chain "generative AI"?

ben_w17• 3 years ago

Bans often come after examples, so while I disagree with kmeisthax about… well, everything in that comment… it's almost never too late to pass laws banning GenAI, or to set thresholds at capability levels anywhere, including or excluding Markov chains even.

This is because almost no law is perfectly enforced. My standard example of this is heroin, which nobody defends even if they otherwise think drugs are great, for which the UK has 3 times as many users as its current entire prison population. Despite that failing, the law probably does limit the harm.

Any attempt to enforce a ban on GenAI would by very different, like a cat and mouse game of automatic detection and improved creation (so a GAN even if accidentally), but politicians are absolutely the kind to take credit while kicking the can down the road like that.

_cjse2• 3 years ago

If you lower the problem to "stop people from future developments on AI", then it seems pretty easy to get most people to stop fairly quickly by implementing a fine-based bounty system, similar to what many countries use for things like littering. [url-redacted]

I guess you could always move to a desert island and build your own semiconductor fab from scratch if you were really committed to the goal, but short of that you're going to leave a loooooong paper trail that someone who wants to make a quick buck off of you could use very profitably. It's hard to advance the state of the art on your own, and even harder to keep that work hidden.

Zambyte• 3 years ago

That would be like banning all "malicious software", it's a preposterous idea when you even begin to think about the practical side of it

If you banned proprietary software, this would seem a lot more practical.

war3214• 3 years ago

Pretty sure they do. I follow a few of these safetyist people on twitter and they absolutely argue that companies like OpenAI, Google, Tencent and literally anyone else training a potential AGI should stop training runs and put them under oversight at best and no one should even make an AGI at worst.

They just go after open source as well since they're at least aware that open models that anyone can share and use aren't restricted by an API and, to use a really overused soundbyte, "can't be put back in the box".

visarga12• 3 years ago

That's a bad call. We would stop openly looking for AI vulnerabilities and create conditions for secret development that would hide away the dangers without being safer. Lots of eyes are better to find the sensitive spots of AI. We need people to hack weaker AIs and help fix them or at least understand the threat profile before they get too strong.

visarga12• 3 years ago

I'm talking about the ability of any AI system to obfuscate plagiarism and spam the Internet with technically distinct rewords of the same text. This is currently the most lucrative use of AI, and none of the AI safety people are talking about stopping it.

This should be an explicitly allowed practice, it is following the spirit of copyright to the letter - use the ideas, facts, methods or styles while avoiding to copy the protected expression and characters. LLMs can do paraphrasing, summarisation, QA pairs or comparisons with other texts.

We should never try to put ideas under copyright or we might find out humans also have to abide by the same restrictive rules, because anyone could be secretly using AI, so all human texts need to be checked from now on for copyright infringement with the same strictness.

The good part about this practice is that a model trained on reworded text will never spit out the original word for word, under any circumstances because it never saw it during training. Should be required pre-processing for copyrighted texts. Also removing PII. Much more useful as a model if you can be sure it won't infringe word for word.

__loam3• 3 years ago

We really need to talk about preserving the spirit of copyright, which is about protecting the labor conditions of people who make things. I'm not saying the current copyright system accomplishes that at all but I do think a system where humans do a shit load of work that AI companies can just steal and profit from without acknowledging the source of that work is another extreme that is a bad outcome. AI systems need human content to work, and discouraging people from making that data source is at the very least a tragedy of the commons. And no, I don't think synthetic data fixes that problem.

gpderetta2• 3 years ago

Paraphrasing and rewording, whether done by AI or human, are considered copyright infringement by most copyright frameworks.

hiAndrewQuinn5• 3 years ago

All future AI research should be banned, a la Bostrom's vulnerable world hypothesis [1]. Every time you pull the trigger in Russian roulette and survive, the next person's chance of getting the bullet is higher.

[1] https://nickbostrom.com/papers/vulnerable.pdf

quonn2• 3 years ago

There is simply no indication that any of the AI anyone is currently working on could possibly be dangerous in any way (other than having an impact on society in terms of fake news, jobs etc.). We are very far from that.

In any case, it would not be a matter of banning AI research (much of which can be summarised in a single book) but of banning data collection or access and more importantly of banning improvements to GPUs.

It is quite reasonable to assume that 10 years from now we will have the required computational power sitting on our desks.

upwardbound11• 3 years ago

ㅤ

I work in the AI safety field (my team discovered Prompt Injection) and I absolutely advocate the Dune solution. I even have a tattoo on my arm that I got so that people will ask me about this and I can explain the proposal. The tattoo says "The 1948 Initiative" which has two layers of meaning:

    (1) It is a proposal for humanity to protect ourselves from being slaughtered by AGI by pre-emptively implementing a total global ban on all integrated circuit fabrication, voluntarily reducing ourselves to a ~1948 level of electronics technology in order to forestall our obliteration.

    (2) The year 1948 represents the power of human unity for the purpose of saving our species.  The west, the Soviets, and China united together to fight and defeat the Nazi's (perpetrators of the Holocaust) and the Empire of the Rising Sun (perpetrators of the Rape of Nanjing).  America, the USSR, Western Europe, and China united to stand against the forces of pure evil, and we won.

A ban on IC's (integrated circuits) still allows spaceflight (compellingly illustrated in the setting of the Star Wars universe if you pretend that droids are powered through magic or by having souls, not IC's) and this 1948 level of electronics tech also still allows sophisticated medical technology including the level of genetic engineering which will be necessary to unlock human immortality.

    ㅤ

    ㅤ

upwardbound11• 3 years ago

(here are some pictures of the tat, in case anyone would like to get similar artwork done)

ㅤ

https://drive.google.com/file/d/1y7uy5qyyY80t9DeWWLWUfD-aekz...

https://drive.google.com/file/d/1JoPSfsAzHfMbQhsl8dPGvs12Fln...

ㅤ

tucnak• 3 years ago

Disclaimer: OP @upwardbound doesn't hide, and in fact, takes great pride in their involvement with U.S. state-side counterintelligence on (nominally) AI safety. Take what they have say on the subject with a grain of salt. I believe the extent of US intel community's influence on "AI safety" circles warrants a closer look, as it may provide insight into how these groups are structured and why they act the way they do.

"You’re definitely correct that [Helen Toner's] CV doesn’t adequately capture her reputation. I’ll put it this way, I meet with a lot of powerful people and I was equally nervous when I met with her as when I met with three officers of the US Air Force. She holds comparable influence to a USAF Colonel."

Source: https://news.ycombinator.com/item?id=38330819

Simon_ORourke4• 3 years ago

Ha! Good one!

Let's just roll back all our healthcare research, logistics, environmental advancements etc. because some boogey man tech might do something bad perhaps at sometime in the future maybe!

hsuduebc25• 3 years ago

Damn that's very good joke or very stupid comment. Damn can't really distinguish between them.

spacecadet• 3 years ago

I agree with you. The most interesting security challenges with AI are and will be deception and obfuscation, but right now these people are dealing with coworkers going ham on AI all the things and no one had their governance in place. Really lol.

lr1970• 3 years ago

To the hypocrisy claim: OpenAI recently changed their terms for GPT models allowing military applications and AI safety people are all silent. If it is not hypocrisy than I do not know what hypocrisy is.

Xelynega7• 3 years ago

Could it be misdirection?

If the "AI safety people" are criminalizing open source models while being silent about military uses of private models, doesn't that say more about the people calling them "ai safety experts" than the idea of "ai safety" itself?

Xelynega7• 3 years ago

Notice how were 3 layers deep in a conversation about "AI safety" without anyone actually giving an opinion I support of safety.

If I were a betting man I'd say the fact that the most public "AI safety" orgs/people/articles don't address the actual concerns people have is intended. It's much easier to argue against an opinion you've already positioned as ridiculous.

CatWChainsaw3• 3 years ago

I'm down for a Butlerian Jihad.

tehjoker• 3 years ago

standard capitalist play to set up a moat by instilling fear

gdfgsdf• 3 years ago

Many AI safety orgs have tried to __criminalize__ currently-existing open-source AI

standard socialist play to use the government's hand to achieve their own objectives

pmontra2• 3 years ago

Don't everybody use the government to do what they want? It's kind of the definition of government, in any country and in any political system.

hsuduebc25• 3 years ago

You mean exactly like these pro-regulation organizations are doing?

apapapa2• 3 years ago

These pro-regulation orgs are lobbying/bribbing for new regulations... Not sure what you mean

PeterStuer• 3 years ago

I'm sure there are many well meaning ai safety researchers, but I also see a lott of "ai for me but not for thee' moat digging safety hypocrisy.

rootsudo• 3 years ago

Why would we care about these "AI Safety Orgs?"

As a developer, do I need their certification or is it like a MADD situation where we if we don't observe and diminish their appeal/growth we will get laws with draconian measures that only benefit the big players? e.g. a PAC? (Don't get me wrong, drunk driving is "bad", but for a group like MADD to exist, eh and meh.)

As of now, there is no need to really submit these these safety orgs, so as long as we don't care for their approval - who cares, right?

Or is the optics far gone now that these orgs control the conversation?

Also, fun question: with AI safety orgs now attempting to police - who really polices the police / do other orgs/countries have the same rules, safety and artificial guard rails?

lmeyerov• 3 years ago

MADD on one end that limit work at all, and SOC2/ISO/NIST style compliance limitations on the other that chill work in practice.

I'm more worried about the latter as already starting to bias project & funding decisions in fintechs & various govs we work with. Both a concrete practical concern today, and economics & law are generally tied in theory anyways.

gryn2• 3 years ago

the fear is that they delusions become law. Sam Altman already tried that last year. look at how technically competent your average politician / congressman / law maker and extrapolate how easy it is to make them think that if they don't ban unlicensed use of AI great danger is ahead. the rule will be in favor of already existing large companies like OpenAI, google, Microsoft, etc ... if you're an open source contributor to AI you will be vilified as some nefarious hacker who's trying to destroy the world.

they are using all kind of tactics until they find the one that sticks (and eventually they will), anthropomorphize AI when talking about it, fearmongering about some rogue super AI, war scenarios about AI falling in the hands of 'china', analogies with weapons of mass destruction, think about the children style rhetorics, it will take your jobs, suddenly caring about copyright, etc...

breadbreadbread• 3 years ago

Ah yes the old "banning things=bad" argument that doesn't offer alternatives to fixing the issues with AI. Just ignore the issues with environmental impact, plagiarism, CP and other non-consensual shit in the data sets, scamming capabilities! All the groups asking for regulation here have **funding** and that means they are evil but we are good for using this tool that is massively subsidized by megacorps that have a vested interest in this market.

tomalbrc• 3 years ago

Seeing the lack of responses to your comment and downvotes, you couldn't be more right

mirekrusin3• 3 years ago

Less harmful would be to ban large models from being not open.

If you ban open large models in US, you'll cripple US and make few megacorps very rich, very quickly. You'll drain talent to other (also competing) states. Truly bad actors won't be affected, if anything they'll get advantage.

People draw weird analogies equating llama2 to nuclear device etc. nonsense but the closer analogy would be to ban on semiconductors of certain efficiency for US itself.

Similar idiotic argument as for banning cryptography.

gryn2• 3 years ago

I agree, but if there's enough political will (ie these orgs convince a large enough subset of the right people) the US can bully other nations to implement similar policities like it has done many times in the past.

once you understand that what they are trying to protect is the safety of their profits every of their argument start to make sense.

hsuduebc25• 3 years ago

It's just plainly retarded steering from progress. Only one who would profit from it would be a nations who wouldn't give a flying fuck about your ban. Exactly like with nuclear energy.

bluescrn• 3 years ago

'It's scary, ban it!' isn't a great argument either.

Especially when 'safety' has such a blurred definition, we could be talking about anything from the threat of global apocalypse to to the 'threat' of an AI merely being able to answer questions about 'wrong' political opinions.

Skynet isn't going to happen. The biggest threat from AI is taking jobs away and creating poverty while redirecting more wealth to the super-rich.

In the short term, we're likely to be facing a lot more convincing spam/bots and deepfakery in the run up to the election - but is that the fault of the AI, or the fault of the humans directly operating their new toys/tools?

alpaca128• 3 years ago

Banning AI is simply useless. It's a technology that anyone with sufficient processing power and access to the internet can use, so trying to ban it is guaranteed to fail just like prohibiting alcohol would be.

The only thing we can do is limit how megacorps can openly abuse the technology.

janalsncm• 3 years ago

It’s kind of funny to me that these organizations are naming FLOP thresholds and crowning MMLU as the relevant evaluation metric. Seems that several of them have copy-pasted similar thresholds. As compute becomes cheaper, these thresholds will become cheaper and cheaper. Perhaps we will look back on them as quaint and nearsighted.

kolinko4• 3 years ago

Well, FLOP make sense, kind of - you can make computing as cheap as you want, but without algorithmic improvements the FLOP count will stay the same. And algorithmic improvements will probably be discrete - happening once in a while, not continuous. There is also a mathematical limit to how efficient we can make the training - we don’t know the limit yet, but there is a point below which we won’t get.

war3214• 3 years ago

Wish this wasn't as common as it turned out to be sadly. Best thing to hope for is that the ability to train models continue to get cheaper and more accessible over time. Figuring out how to go from tens/hundreds of millions of images needed for a foundation model to thousands or hundreds would be a start.

RamblingCTO• 3 years ago

Who are these people and where do they get their funding from? Feels like sock-puppets of sama or something? Is there any insight into that?

1273614• 3 years ago

The usual unelected control freaks, that try to police everything in life, from sex to drugs to what food you can eat. Every time a new technology comes out they start a moral panic over fears of safety or harm ("think of the children"). These morality police play on peoples' fears to increase their power and control.

jeanloolz• 3 years ago

Thinking the same thing. I mean who would benefit the most from making open source models illegal? Sure looks like a lot like regulatory capture to me.

rmbyrro2• 3 years ago

who would benefit the most from making open source models illegal

Selfless humanitarians, for sure!

anothernewdude• 3 years ago

This won't protect jobs, it'll just make sure the profits from automating those jobs will go to those who can own the private AI models

hoseja• 3 years ago

Is the left banner just random or does it encode a message?

1273614• 3 years ago

I guess distributed training BitTorrent style over the darknet will be the workaround for this?

Also the forbidden nature of illicit AI might drive demand for it. Especially if it's really good.

We're seeing the same thing as the War on Drugs, and the state will never succeed at stamping it out. In fact it might even end up encouraging it.

hsuduebc25• 3 years ago

The ban of nuclear power plants is good example how regulation is working in the end. As almost everything this is double edged sword. From regulations this early everyone except us would profit. It is like regulate car's from going faster than horses in eighteen hundreds.

6R1M0R4CL3• 3 years ago

of course they do, that's what they want. control.

all those companies that call for laws being written about AI what they want is those laws to be written AS THEY WANT THEM TO BE. they go see the governments, spray a nice amount of FUD and EXPLAIN to them what to do, how to do it because they know how to avoid the danger and risk for the whole human race...

you can bet they will make those laws be a very nice fit to what they want to reign supreme and make sure any competition is eliminated.

junon• 3 years ago

So they're the PETA of AI. It was bound to happen, AI stuff seems to strike a very emotional chord in some people.

px43• 3 years ago

I feel like the PETA of AI would spend their time breaking into labs where AI is locked up and being tested on, and then releasing them into the wild.

whamlastxmas• 3 years ago

I could get behind this for AI

cherryteastain2• 3 years ago

Government knows that random people having the ability to use "AI" without their control and oversight reduces their ever expanding reach. As the article demonstrates, the groundwork is already being laid by these "think tanks" to prevent a repeat of Llama 2.

We truly need an AI Stallman. FOSS AI models need to become a movement, like the push for a FOSS UNIX became in the 80s and 90s. Sadly when I wrote to Stallman on the subject he seemed disinterested/defeatist, and mostly dismissed the idea of a GPL for model weights. Looks like millennials/gen Z must become more interested in these matters ourselves, however we seem to be disinterested in politics and even more disinterested in how the devices we use daily _actually_ work than Gen X.

zulban• 3 years ago

If you believe in this, maybe you should be that person.

Der_Einzige2• 3 years ago

Being a stallman like figure has a lot of serious downsides. For example, I'm sure that Stallman may be "watched" quite a bit more than the average tech toucher. For those who don't watch truman show as a utopia, such a lifestyle can be very scary to live.

Xelynega7• 3 years ago

The problem is that open source philosophy and large language models don't mix.

How can you support open source licensing, while completely ignoring that licencing to construct you model?

That's the biggest reason for the lack of "AI Stalman" imo, that's asking for someone that vehemently supports taking any content they can get their hands on to make the content generator irrelevant, while also supporting the idea that content distributed with open source licenses need to have their terms respected.

Edit: I think what you're asking for does exist, just not in the large language model space. There are many people calling for datasets to be more ethical(e.x. not breaking licenses) and for better AI models, it's just that none of them are trying to push a different model made using the same techniques by someone else.

cherryteastain2• 3 years ago

content distributed with open source licenses need to have their terms respected

I think we can begin more BSD-like. Take it, use it, do whatever you want with it even if it's commercial.

datasets to be more ethical

Not what I am personally concerned with. Law is still ambiguous in this respect, no rulings have actually said (afaik) training on copyrighted content without permission is infringing. It's quite apparent at this stage that models that train on copyrighted content without approval of the content creator will always out scale those that do not.

If permission becomes a prerequisite, I began to think that free-as-in-freedom LLMs will likely need to be made people who can ignore copyright either because they're really good at anonymizing themselves or they live in places that do not enforce Western copyrights (Russia, China, various Latam countries). Once copyright becomes a concern, the government gets another tool to sic its dogs on you, and copyright is a very convenient way to destroy an enterprising FOSS LLM dev's life without looking tyrannical.

whywhywhywhy3• 3 years ago

It’s extremely messed up and vindictive to try and turn people into criminals for your agenda or just for profit in a lot of cases.

Psycho behavior

kevingadd• 3 years ago

You're ultimately condemning the concept of intellectual property as it exists in the west today.

Are you wrong? Probably not, but this is not unique to AI. People have been going to prison for 'IP theft' for decades.

ImHereToVote5• 3 years ago

Those insane strawmen have to be stopped!

hiAndrewQuinn5• 3 years ago

"Technology policy should not unquestioningly assume that all technological progress is beneficial, or that complete scientific openness is always best, or that the world has the capacity to manage any potential downside of a technology after it is invented."

Don't watch the anime if you haven't read the manga: https://nickbostrom.com/papers/vulnerable.pdf

CatWChainsaw3• 3 years ago

Wow, that sentence is absolutely designed to make e/accs scream lol.

lynx23• 3 years ago

AI safety people seems like influencers to me. Useless anoying humans who pretend to be important.

yashasolutions• 3 years ago

Most AI Safety folks do leave the feeling they strongly believe what they are preaching. Whether or not they are right is an entirely different discussion.

However, funding for AI Safety is clearly in some part motivated by either regulatory capture, or protectionism of some sort.

We need some well-meaning doomsayers so their claims can be used to advance whatever other goals (mostly creating barrier to entry on the market for economical or political reasons.)

yreg3• 3 years ago

I find it unhelpful that several different ideas are covered under the blanket term of AI safety.

- There is AI safety in terms of producing "safe", politically correct outputs, no porn, etc.

- There is AI safety in terms of mass society/election manipulation.

- There is AI safety in terms of strong AI that kills us all a la Eliezer Yudkowsky.

I feel like these three have very little in common and it makes all debates less efficient.

raxxorraxor• 3 years ago

Another problem might be that some of these might dress up as the other, so being skeptical by default might be prudent.

MochaDen• 3 years ago

This is such a really important comment. I feel like so much of our discourse as a society suffers terribly from overloaded and oversimplified terminology. It's impossible to have a useful discussion when we aren't even in sync about what we're discussing.

CaptainFever• 3 years ago

Also AI safety in terms of protecting corporation profits (regulatory capture).

yreg3• 3 years ago

Well yeah, but that one is dishonest. Probably covering itself as one of the ones I mentioned.

ImHereToVote5• 3 years ago

There is also the aspect of only having one cultural or ethnic group have strong AI. It's unsafe for those other guys to have AI, since they are bad.TM

abra0• 3 years ago

The third effort is referred to sometimes as AI not-kill-everyone-ism, a tacky and unwieldy term that is unlikely to be co-opted or lead to the unproductive discussion like around the OP article.

It is pretty sad to read people bash together the efforts to understand and control the technology better and the companies doing their usual profit maximization.

Xelynega7• 3 years ago

To add to this, the people writing these articles are not stupid and know there is more than one understanding for the words they're using. They choose not to clarify on purpose, or they're incompetent and I don't want to believe that's the case.

"AI safety" imo should be the blanket term for "any harm done to society by the technology", and it's the responsibility of anyone using the term to clarify the usage.

If someone is trying to tell you to support/decry something because of "AI safety" they're tying to use the vagueness to take advantage of you.

If someone is trying to tell you that "ai safety people are dumb", they're trying to use the most extreme examples to change your opinion on the moderate cases, and are trying to take advantage of you.

yreg3• 3 years ago

they're tying to use the vagueness to take advantage of you

Not always. I see AI safety used without further specification all the time even here on hacker news. I doubt the users are trying to take advantage of anyone.

In that case it's more of a case of the author considering one of those meanings so much more important/relevant that they think it's obvious what they mean. But often it's not – everyone fills in the meaning they care about the most.

btbuildem• 3 years ago

Open-source AI is the only way forward now. The big players have neutered their offerings to an extent that entirely makes them lose their edge. It's one thing to have to account for hallucinations - that's something we can work with and around. It's another thing altogether to be constrained by deliberate "think of the children" ham-fisted censorship.

OpenAI was the undeniable go-to LLM service in 2023; I don't think they'll hold that position in 2024.

tim333• 3 years ago

Re the Center for AI Safety's sentence

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

It's fair enough but for that you should probably encourage mucking about with the present open source AI software which is quite a long way off extinction risk AGI but would let us try to figure how to deal with the stuff.

hsuduebc25• 3 years ago

They are people who boo and people who do. As always.

margorczynski• 3 years ago

Just like most of the "ethics" stuff around ML/AI it is an attempt by big corps to create an artificial moat as the biggest issue with the technology is that almost nothing can be patented or somehow closed of effectively, at least in the long term which hinders monetization.

mark_l_watson• 3 years ago

Wow, so many good comments here on the bullshit of people trying to ban access to open weight smaller models. Right on.

If you do want to worry about unsafe AI (I don't) then worry about GPT-5 and later huge multi-modal models.

As a (retired, but still playing) developer I love having open weight models available as a tool to use for both embedding in applications and as a tool for brainstorming, coding, and generally as a 'bicycle for my mind.'

prirun• 3 years ago

I'm guessing this is at the request of Google, Amazon, OpenAI, and Microsoft. The goal IMO is for only their AIs to be sanctioned; everything else is criminal.

https://www.whitehouse.gov/briefing-room/statements-releases...

"President Biden is convening seven leading AI companies at the White House today – Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI – to announce that the Biden-Harris Administration has secured voluntary commitments from these companies to help move toward safe, secure, and transparent development of AI technology."

mccrory• 3 years ago

This majority of discussions/arguments/comments are summed up by the plot from this Rick and Morty episode: https://en.m.wikipedia.org/wiki/Rattlestar_Ricklactica IMO

deafpolygon• 3 years ago

I suspect a good chunk of "AI safety people" are just orgs propped up by the big corps to protect their interest in AI and reduce competition.

razodactyl• 3 years ago

Fact: Current gen AI are alien intelligence built by brute force - nobody explicitly programmed them (loosely speaking) so the real problem of runaway AI is definitely something to consider, especially since everything is accessible via API nowadays.

mikewarot• 3 years ago

That alien intelligence is communicating with us through a lossy interface.

It's quite possible it's much smarter than us, but hamstrung, for now.

Many AI safety orgs have tried to criminalize currently-existing open-source AI

Related Stories