Machine Unlearning in 2024

https://ai.stanford.edu/~kzliu/blog/unlearning

by ignoramous • 2 years ago

328 94 2 years ago

ai.stanford.edu

dataflow• 2 years ago

However, RTBF wasn’t really proposed with machine learning in mind. In 2014, policymakers wouldn’t have predicted that deep learning will be a giant hodgepodge of data & compute

Eh? Weren't deep learning and big data already things in 2014? Pretty sure everyone understood ML models would have a tough time and they still wanted RTBF.

peteradio• 2 years ago

I don't know if people anticipated contemporary parroting behavior over huge datasets. Modern well funded models can recall an obscure persons home address buried deep into the training set. I guess the techniques described might be presented to the European audience in an attempt to maintain access to their data/and or market for sales. I hope they fail.

isodev4• 2 years ago

Of course, it’s not a regulation issue. The technology was introduced to users before it was ready. The very nature of training without opt-in consent or mechanism of being forgotten are all issues that should have been addressed before trying to make a keyboard with a special copilot button.

spennant2• 2 years ago

Agreed. The media and advertising industry was most definitely leveraging cookie-level data for building attribution and targeting models. As soon as the EU established that this data was “personal data”, as it could, theoretically, be tied back to individual citizens, there were questions about the models. Namely “Would they have to be rebuilt after every RTBF request?” Needless to say, no one in the industry really wanted to address the question, as the wrong answer would essentially shut down a very profitable practice.

Aerroon• 2 years ago

More likely: the wrong answer would've shut out a profitable market rather than the practice. The EU is not the world. Anthropic seems to not mind blocking the EU for example.

startupsfail• 2 years ago

RTBF was introduced to solve a specific issue, no?

Politicians and their lobbyist friends could no longer remove materials linking them to their misdeeds as the first Google Search link associated with their names. Hence RTBF.

Now, there’s similar issue with AI. Models are progressing towards being factual, useful and reliable.

indigovole• 2 years ago

GDPR and RTBF were formulated around the fears of data collection by the Stasi and other organizations. They were not formulated around easing the burdens of future entrepreneurs, but about mitigating the damage they might cause. Europeans were concerned about real harms that living people had experienced, not about enabling AGI or targeted advertising or digital personal assistants.

We have posts here at least weekly from people cut off from their services, and their work along with them, because of bad inference, bad data, and inability to update metadata based purely on BigGo routine automation and indifference to individual harm. Imagine the scale that such damage will take when this automation and indifference to individual harm are structured around repositories from which data cannot be deleted, cannot be corrected.

hooby• 2 years ago

I'm pretty sure that the policymakers did NOT understand ML models in 2014 - and still do NOT understand it today.

I also don't think that they care. They don't care that ML is a hodgepodge of data & compute, and they don't care how hard it is to remove data from a model.

They didn't care about the ease or difficulty of removing data from more traditional types of knowledge storage either - like search indexes, database backups and whatnot.

RTBF was not proposed with any specific technology in mind. What they had in mind, was to try and give individuals a tool, to keep their private information private. Like, if you have a private, unlisted phone number, and that number somehow ends up on the call-list of some pollster firm, you can force that firm to delete your number so that they can't call you anymore.

The idea is, that if your private phone number (or similar data) ends up being shared or sold without your consent - you can try to undo the damage.

In practice it might still be easier to get a new number, than to have your leaked one erased... but not all private data is exchangeable like that.

negative_person2• 2 years ago

Why should we try to unlearn "bad" behaviours from AI?

There is no AGI without violence, its part of being free thinking and self survival.

But also by knowing that launching a first strike by a drunk president was a bad idea we averted a war because of a few people, AI needs to understand consequences.

It seems futile to try and hide "bad" from AI.

andy993• 2 years ago

This is presumably about a chatbot though, not AGI, so it's basically a way of limiting what they say. (Not a way that I expect to succeed)

Cheer2171• 2 years ago

So you have a problem with supervised learning like spam classifiers?

williamtrask2• 2 years ago

Because we can get AI related technologies to do things living creatures can’t, like provably forget things. And when it benefits us, we should.

Personal opinion, but I think AGI is a good heuristic to build against but in the end we’ll pivot away. Sort of like how birds were a good heuristic for human flight, but modern planes don’t flap their wings and greatly exceed bird capabilities in many ways.

Attribution for every prediction and deletion seem like prime examples of things which would break the analogy of AI/AGI with something more economically and politically compelling/competitive.

negative_person2• 2 years ago

Can you point to any behaviour in human beings you'd unlearn if theyd also forget the consequences?

We spend billions trying to predict human behaviour and yet we are surprised everyday, "AGI" will be no simpler. We just have to hope the dataset was aligned so the consequences are understood, and find a way to contain models that don't.

doubloon• 2 years ago

AGI would not beGI unless it could change its mind after realizing its wrong about something

5424583• 2 years ago

I disagree. People with anterograde amnesia still possess general intelligence.

sk11001• 2 years ago

The point is to build things that are useful, not to attempt to replicate science fiction literature.

szundi• 2 years ago

Thanks but no violent AGIs thanks

5424583• 2 years ago

There is no AGI without violence, its part of being free thinking and self survival.

I disagree. Are committed pacifists not in possession of general intelligence?

imtringued• 2 years ago

You seem to be ignoring the potential to use this to improve the performance of LLMs. If you can unlearn wrong answers you can ask the model using any scoring mechanism to check for correctness instead of scoring for token for token similarity to the prescribed answer.

affgrff2• 2 years ago

Maybe it all boils down to copyright. Having a method that believably removes the capacity to generate copyrighted results might give you some advantage with respect to some legislation.

wongarsu• 2 years ago

Also if you build some sort of search engine using an LLM governments will expect you to be able to remove websites or knowledge of certain websites for legal reasons (DMCA, right to be forgotten, etc).

numpad0• 2 years ago

They are just trying to find a way to plausibly declare successful removal of copyrighted and/or illegal material without discarding weights.

GPT-4 class models reportedly costs $10-100m to train, and that's too much to throw away for Harry Potter or Russian child porn scrapes that could later reproduce verbatim despite representing <0.1ppb or whatever minuscule part of dataset.

wruza• 2 years ago

There is no AGI without violence, its part of being free thinking and self survival.

Self survival idea is a part of natural selection, AGI doesn't have to have it. Maybe the problem is we are the only template to build AGI from, but that's not inherent to "I" in any way. Otoh, lack of self preservation can make animals even more ferocious. Also there's a reason they often leave a retreat path in warzones.

Long story short it's not that straightforward, so I sort of agree cause it's an uncharted defaults-lacking territory we'll have to explore. "Unlearn bad" is as naive as not telling your kids about sex and drugs.

surfingdino2• 2 years ago

AI has no concept of children, family, or nation. It doesn't have parental love or offspring protection instinct. Faced with danger to its children it cannot choose between fighting or sacrificing itself in order to protect others. What it is good at is capturing value through destruction of value generated by existing business models; it does it by perpetrating mass theft of other people's IP.

Jaygles• 2 years ago

Because corporations won't buy the fancy chat bot if there's a chance it will occasionally use slurs in it's interactions with their customers.

cwillu2• 2 years ago

“to edit away undesired things like private data, stale knowledge, copyrighted materials, toxic/unsafe content, dangerous capabilities, and misinformation, without retraining models from scratch”

To say nothing of unlearning those safeguards and/or “safeguards”.

ben_w2• 2 years ago

It sounds like you're mistakenly grouping together three very different methods of changing an AI's behaviour.

You have some model, M™, which can do Stuff. Some of the Stuff is, by your personal standards Bad (I don't care what your standard is, roll with this).

You have three solutions:

1) Bolt on a post-processor which takes the output of M™, and if the output is detectably Bad, you censor it.

Failure mode: this is trivial to remove, just delete the post-processor.

Analogy: put secret documents into a folder called "secret do not read".

2) Retrain the weights within M™ to have a similar effect as 1.

Failure mode: this is still fairly easy to remove, but will require re-training to get there. Why? Because the weights containing this information are not completely zeroed-out by this process.

Analogy: how and why "un-deletion" is possible on file systems.

3) Find and eliminate the weights within M™ that lead to the Bad output.

Analogy: "secure deletion" involves overwriting files with random data before unlinking them, possibly several times if it's a spinning disk.

People are still doing research on 3 to make sure that it actually happens, what with it being of very high importance for a lot of different reasons including legal obligation.

andy993• 2 years ago

Until we have a very different method of actually controlling LLM behavior, 1 is the only feasible one.

Your framing only makes sense when "Bad" is something so bad that we can't bear its existence, as opposed to just "commercially bad" where it shouldn't behave that way with an end user. In the latter, your choice 1 - imposing external guardrails - is fine. I'm not aware of anything LLMs can do that fits in the former category.

cwillu2• 2 years ago

I think you mistakenly replied to my comment instead of one that made some sort of grouping?

Alternatively, you're assuming that because there is some possible technique that can't be reversed, it's no longer useful to remove the effects of techniques that _can_ be reversed?

nullc3• 2 years ago

I've wondered before if it was possible to unlearn facts, but retain the general "reasoning" capability that came from being trained on the facts, then dimensionality reduce the model.

andy993• 2 years ago

If you think of knowledge as a (knowledge) graph, it seems there would be some nodes with low centrality that you could drop without much effect, and other key ones that would have a bigger impact if lost.

huygens6363• 2 years ago

Yes, me too. If it could somehow remember the “structure” instead of the instantiation. More “relationships between types of token relationships” instead of “relationships between tokens”.

Brian_K_White2• 2 years ago

I don't know about in AI, but it seems like that is what humans do.

We remember some facts but I know at least I have had a lot of facts pass through me and only leave their effects.

I once had some facts, did some reasoning, arrived at a conclusion, and only retained the conclusion and enough of the reasoning to identify other contexts where the same reasoning should apply. I no longer have the facts, I simply trust my earlier selfs process of reasoning, and even that isn't actually trust or faith because I also still reason about new things today and observe the process.

But I also evolve. I don't only trust a former reasoning unchanging forever. It's just that when I do revisit something and basically "reproduce the other scientists work" even if I arrive a different conclusion today, I'm generally still ok with the earlier me's reasoning and conclusion. It stands up as reasonable, and the new conclusion is usually just tuned a little, not wildly opposite. Or some things do change radically but I always knew they might, like in the process of self discovery you try a lot of opposite things.

Getting a little away from the point but the point is I think the way we ourselves develop answer-generating-rules is very much by retaining only the results (the developed rules) and not all the facts and steps of the work, at least much of the time. Certainly we remember some justifying / exemplifying facts to explain some things we do.

mr_toad2• 2 years ago

How much reasoning capability LLM’s have is up for debate.

With a true AGI you could just tell it to keep people’s personal information confidential and expect that it would understand that instruction.

actionfromafar• 2 years ago

It could understand it and try to comply but still fail to understand where it would leak data which can later be corroborated to someone. This is at least what commonly happens with human AGis.

karma_pharmer3• 2 years ago

This presumes that LLMs actually contain "reasoning" capability other than "the facts" and simple interpolation between them.

It is far from clear that this presumption holds.

Ingesting more text than a human could read in several lifetimes produces the ability to interpolate answers to a surprisingly large range of questions. This is not intuitive to humans, because we've never met anybody who could possibly read this much raw text, so we mistake this level of interpolation for reasoning since we've never met anybody who could possibly memorize-and-interpolate so much. The only thing we've ever seen that can answer questions like this is a human using reasoning capabilities, so we assume that's what this thing on the other side of the screen must be doing. Like chimpanzees mistaking their own reflection in a mirror for another ape.

This is the other "bitter lesson" of ML.

But if you spend enough time playing with these models you start to figure out what questions to ask to make them look foolish. And that is totally fair game for the Turing Test. Remember, there is no time limit on the Turing Test. However there is a strict requirement that the machine under test cannot be serviced, modified, or updated while the test is underway -- and because of this, nothing that OpenAI has produced is capable of even taking the test. We know that OpenAI tweaks and tunes their models whenever they please, as many times a day as they like, and that they use discussions here on HN to feed that process. So stick to the models you can download.

It's uncontroversial to say that this sort of "massive interpolation using superhuman text ingestion" is exactly how modern machine translation models work, and they work extremely well. LLMs were created by taking a machine translation model, throwing away half of it, and then fiddling with the leftovers.

nullc3• 2 years ago

There is clearly a degree of abstraction-- consider you can make up some game which creates words the LLM has never seen before and assigns them meaning, then ask the LLM to reason about them within the logic of the game and it will do so at least somewhat successfully.

(much worse than it does stuff it's seen before, for sure, but that it does it at all shows there is some abstraction)

Now if that qualifies as "reasoning" is another question, but it may be a metaphysical one with little value in making the world a better place. :P

Whatever we call it there clearly is some amount of emergent abstraction in the models which is useful for at least some applications (if many fewer than the hype suggests). Can that abstraction be isolated from the factual data that went in to construct it? If so then perhaps we could have smaller models with better performance or construct ways to amplify that "operating over abstraction" until it did meet whatever bar you'd require to call it "reasoning", or at least become more useful along the way.

motohagiography• 2 years ago

seems like there is a basic problem where if you specify something to be unlearned, it could still be re-learned by inference and prompting. the solution may not be in filtering the proscribed facts or data itself, but in the weights and incentives that form a final layer of reasoning. Look at "safe" models now like google's last launch, where the results were often unsatisfying, as clearly we don't want truthful models yet, but we want ones that enable our ability to develop them further, which for now means not selecting out by antagonizing other social stakeholders.

maybe we can encode and weight some principle of the models having been created by something external, with some loosely defined examples they can refer to as a way to evaluate what they return, then ones that don't yield those results cease to be used, where the ones that find a way to align will get reused to train others. there will absolutely be bad ones, but in aggregate they should produce something more desirable, and if they really go off the rails, just send a meteor. the argument in how models can "unlearn" will be between those who favour incentives and those who favour rules- likely, incentives for ones I create, but rules for everyone elses'.

musicale• 2 years ago

It is unsurprising that a system trained on human-generated content might end up encoding implicit bias, toxicity, and negative goals. And the more powerful and general-purpose a system is, the more suitable it is for a wide range of powerfully negative purposes.

Neither specializing the model nor filtering its output seems to have worked reliably in practice.

gotoeleven• 2 years ago

My new startup includes a pitchfork wielding mob in the ML training loop.

avi_vallarapu3• 2 years ago

We need to consider the practicality of unlearning methods in real-world applications and the legal acceptance of the same.

Given current technology and what advancements are needed to make Unlearning more possible, probably there should be a time-to-unlearn kind of an acceptable agreement that allows organizations to retrain or tune the response that does not involve any response from the to-be-unlearned copyright content.

Ultimately, legal acceptance for unlearning may be all about deleting the data set that is part of any kind of violations from the training data set. It may be very challenging to otherwise prove legally through the proposed unlearning techniques, that the model does not produce any type of response involving the private data.

The actual data set contains the private data violating privacy or copyright, and the model is trained on it, period. This means, it must involve retraining by deleting the documents/data to be unlearned.

isodev4• 2 years ago

a time-to-unlearn kind of an acceptable agreement

Why put the burden to end users? I think the technology should allow for unlearning and even "never learn about me in any future models and derivative models".

avi_vallarapu3• 2 years ago

No technology can guarantee 100% unlearning, and the only 100% guarantee is when the data is deleted before the model is retrained. Legally, even 99.99% accuracy may not be acceptable, but, only 100%.

Vampiero• 2 years ago

The technology is on par with a Markov chain that's grown a little too much. It has no notion of "you", not in the conventional sense at least. Putting the infrastructure in place to allow people (and things) to be blacklisted from training is all you can really do, and even then it's a massive effort. The current models are not trained in such a way that you can do this without starting over from scratch.

beeboobaa37• 2 years ago

How to deal with "unlearning" is the problem of the org running the illegal models. If I have submitted a gdpr deletion request you better honor it. If it turns out you stole copyrighted content you should get punished for that. No one cares how much it might cost you to retrain your models. You put yourself in that situation to begin with.

avi_vallarapu3• 2 years ago

Exactly, I think is where it leads to eventually. And that is what I my original comment meant as well. "Delete it" rather than using some more techniques to "unlearn it", unless you claim the unlearning is 100% accurate.

visarga• 2 years ago

No one cares how much it might cost you to retrain your models.

Playing tough? But it's misguided. "No one cares how much it might cost you to fix the damn internet"

If you wanted to retro-fix facts, even if that could be achieved on a trained model, it would still get back by way of RAG or web search. But we don't ask pure LLMs for facts and news unless we are stupid.

If someone wanted to pirate a content it would be easier to use Google search or torrents than generative AI. It would be faster, cheaper and higher quality. AIs move slow, are expensive, rate limited and lossy. AI providers have in-built checks to prevent copyright infringement.

If someone wanted to build something dangerous, it would be easier to hire a specialist than to chatGPT their way into it. All LLMs know is also on Google Search. Achieve security by cleaning the internet first.

The answer to all AI data issues - PII, Copyright, Dangerous Information - is coming back to the issue of Google search offering links to it, and websites hosting this information online. You can't fix AI without fixing the internet.

friendzis• 2 years ago

We need to consider the practicality of unlearning methods in real-world applications and the legal acceptance of the same. probably there should be a time-to-unlearn kind of an acceptable agreement

A very important distinction is between data storage and data use/dissemination. Your comment hints at "use current model until retrained is available and validated", which is an extremely dangerous idea.

Remember old times of music albums distributed over physical media. Suppose a publisher creates a mix, stocks shelves with album and it becomes known that one of the tracks is not properly licensed. It would be expected that it takes some time to execute distribution shutdown: distribute order, clean up shelves, etc. However, time for another production run with a modified tracklist would be entirely the problem of the publisher in question.

The window for time-to-unlearn should only depend on practicality of stopping information dissemination, not getting updated source ready. Otherwise companies will simply wait for model to be retrained on a single 1080 and call it a day, which would effectively nullify the law.

aidenn0• 2 years ago

I think "unlearning" is not the actual goal; we don't want the model to stick its proverbial head in the sand. Being unaware of racism is different from not producing racist content (and, in fact, one could argue that it is necessary to know about racism if one wishes to inhibit producing racist content; I remember in elementary school certain kids thought it would be funny to teach one of the special-ed kids to parrot offensive sentences).

krono• 2 years ago

Say you tell me you want a red sphere. Taken at face value, you show a prejudice for red sphere's and discriminate against all other coloured shapes.

We've all had to dance that dance with ChatGPT by now, where you ask for something perfectly ordinary, but receive a response telling you off for even daring to think like that, until eventually you manage to formulate the prompt in a way that it likes with just the right context and winner vocabulary + grammar, and finally the damned thing gives you the info you want without so much as any gaslighting or snarky insults hiding in the answer!

It doesn't understand racism, it simply evaluates certain combinations of things according to how it was set up to do.

greenavocado• 2 years ago

Please use the correct terminology: censorship

danielmarkbruce• 2 years ago

If company X wants their model to say/not say Y based on ideology, they aren't stopping anyone saying anything. They are stopping their own model saying something. The fact that I don't go around screaming nasty things about some group doesn't make me against free speech.

It's censorship to try to stop people producing models as they see fit.

62951413• 2 years ago

The prolefeed explains that deep duckspeaking is doubleplusgood. Nothing to see here, citizen.

qbit42• 2 years ago

I don't think that's a fair characterization. If a user requests a company to stop using their data, ML unlearning allows the company to do so without retraining their models from scratch.

Dylan168074• 2 years ago

Is it censorship to not include every piece of text you can possibly find into your training dataset?

What's the difference between making that choice versus removing it from the model later?

surfingdino2• 2 years ago

How about a radial approach? How about not ingesting all content but only that which is explicitly marked as available for model-building purposes?

xg152• 2 years ago

What I don't get about the DP approach is how this would be reconciled with the "exact" question-answering functionality of LLMs.

DP makes perfect sense if all I care about is low-resolution statistical metrics or distributions of something and not the exact values - the entire purpose of DP is to prevent reconstructing the exact values.

However, the expectation for LLMs is usually to ask a question (or request a task) and get an exact value as a response: If you ask "What's the phone number of John Smith?" the model will either tell you it doesn't know or it will answer you with an actual phone number (real or hallucinated). It will not tell you "the number is with 83% probability somewhere in New Jersey".

So if the model is trained with DP, then either the data is scrambled enough that the it won't be able to return any kind of reliably correct data, effectively making it useless - or it's not scrambled enough, so that the model can successfully reconstuct data despite the scrambling process, effectively making the DP step useless.

Or in other words, the OP defines "DP unlearning" as:

The intuition is that if an adversary cannot (reliably) tell apart the models, then it is as if this data point has never been learned—thus no need to unlearn.

However, if my original model truthfully returns John Smith's phone number on request and the "unlearned" model must not be distinguishable by an outside observer from the original model, then the "unlearned" model will also return the phone number. While I could say that "technically" the model has never seen the phone number in the training data due to my DP scrambling, this doesn't solve the practical problem why the unlearning was requested in the first place, namely that John Smith doesn't want the model to return his phone number. He could probably care less about the specific details of the training process.

So then, how would DP help here?

JKCalhoun• 2 years ago

I don't know — the post, reading the comments here, I am a little worried for the "sanity" of our AI that have been trained, untrained, retrained like a pawn in some kind of Cold War spy novel.

kombookcha• 2 years ago

It's fine, the LLM AIs we have now are just fancy versions of autocorrect. They, and other LMs, guess at statistically probable words/datapoints, and because they don't understand context, you might need to put your thumb on the scales to make the output actually be useful. They're at best very janky tools as soon as you're working with things that require context that isn't easily contained in some kind of confined area of work.

Currently we are seeing the phenomenon 'habsburg AI' where AI's consume their own outputs as training data, which rapidly deteriorates their ability to actually be useful for much of anything.

The thing is that there literally isn't enough human-made data to keep feeding them (they already ate the entire internet), so if you both want to continue ramping their intake of data and you also don't want them to get rapidly weird and completely useless, you pretty much have to get in there with elbow grease. Removing or deprioritizing data that's tripping up the model is one of the few ways you can do human-assisted refinement of these things.

The sooner we all face the music that these things aren't magical truth machines, have a long way to go and there is no guaranteed rate of growth, the sooner this hype cycle can end.

joshhansen• 2 years ago

"Eternal Sunshine of the Spotless Mind"

The erasure of knowledge is a troubling occupation

thomastjeffery• 2 years ago

So "unlearning" is yet another overconfident implementation of an impossible task. I guess that's par for the course.

My favorite part is where literally pretending is presented as a serious option. Wow. Just..wow.

Machine Unlearning in 2024

Related Stories