HNGlance

Top Best New Show

Locating and Editing Factual Associations in GPT

https://rome.baulab.info/

by codeulike • 3 years ago

1 1 3 years ago

RO

rome.baulab.info

Related Stories

Loading related stories...

Comments

1

codeulike• 3 years ago

Knowing differs from saying: uttering words by rote is different from knowing a fact, because knowledge of a fact generalizes across contexts. In this project, we show that factual knowledge within GPT also corresponds to a localized computation that can be directly edited. For example, we can make a small change to a small set of the weights of GPT-J to teach it the counterfactual "Eiffel Tower is located in the city of Rome." Rather than merely regurgitating the new sentence, it will generalize that specific counterfactual knowledge and apply it in very different linguistic contexts.

Another demonstration that LLMs are indeed making internal models

Source preview

rome.baulab.info

HNGlance

© 2024-2026 HN Glance. Story content belongs to its original sources.

Privacy
Terms