Dola Decoding by Contrasting Layers Improves Factuality in Large Language Models

by johnsutor • 2 years ago

58 43 2 years ago

photonthug• 2 years ago

Just call it correctness. Hallucination as an alternative to incorrect is fine for marketing I guess but factuality is especially awkward besides being pretty Orwellian.

skdotdan• 2 years ago

The way I see it hallucination and factual error are particular cases of incorrect answer.

HeatrayEnjoyer2• 2 years ago

Orwellian? Maybe, but in the same way we teach our children what is true and how to determine what is true.

We want to raise these pseudo-humans to be useful upstanding members of society. Knowing fact from opinion, knowing right from wrong, knowing what is real and what is imagination, are important for any intelligence. Otherwise our silicon-children will grow up to be dumb, harmful, or both, while being trillions in number.

prometheus764• 2 years ago

And that goes back to the question I asked above: are you talking about "what is real" from a philosophical "realism" point of view, or from a philosophical "nominalism" point of view?

Realism posits that objects have intrinsic meaning that we apprehend through attention.

Nominalism posits that we cannot apprehend reality directly, but only through our minds and through language. That we only have a second-order experience of reality. Therefore, all language only has meaning because of consensus, so if we change the consensus of meaning around language, we are actually changing reality because reality is mediated through language.

These are obviously very compressed definitions of these views, but the question remains.

This conversation about AI "hallucinations" seem to point at this question. "We want AI to say true things." True to what? True to reality? Or true to language? When we ask AI a question, AI only knows how to answer the question using grammar that is probabilistically the most likely. That has no tie to "reality", but as soon as you start asking "well then what is reality that we want to map AI to?" the question gets quite slippery.

My contention is that AI, as its responses are curated by people, will only reflect the idiosyncratic worldviews of those doing the pruning.

Nevermark• 2 years ago

I prefer the word “confabulate”.

When humans fill in knowledge they don’t actually have, but think we do, we call it confabulating.

We all confabulate at a low level, because it is intrinsic to how we store and recall memories. Our memories are compressed summaries whose details we fill out with defaults and guesswork subconsciously as we “remember”. This is why memories are rarely perfectly accurate.

Some people confabulate more than others. And we all confabulate to a greater or lesser degree based on variable circumstances, emotions, motivation, fatigue and other mental states.

“Hallucinations” on the other hand are what happens when our sensory processing becomes unmoored from actual memory or real sensory constraints. The brain creates an interpretation based on internal feedback without normal correction, drifting into false sensory experiences that are not actually reflective of reality.

Dreams are a natural form of this where nothing we are experiencing is actually really happening.

One is false memories, the other is false experiences.

prometheus764• 2 years ago

So will LLMs ultimately become realists, or nominalists?

soist16• 2 years ago

LLMs can be whatever labels people choose to attribute to the system executing the instructions to generate "answers". It is fundamentally a category error to attribute any meaning to whatever arithmetic operations the hardware is executing because neither the hardware in the data center nor the software have any personal characteristics other than what people erroneously attribute to them because of confused ontologies and metaphysics about computers and software.

HeatrayEnjoyer2• 2 years ago

At which point would such attributions be accurate? Humans are fundamentally just computers too. A different medium, but still transforming electrical signals.

totetsu• 2 years ago

exploiting the fact that factual knowledge in an LLMs has generally been shown to be localized to particular transformer layers

This is surprising

dinobones• 2 years ago

Why is this surprising?

It makes sense that “facts” exist in earlier layers, then these become more abstract as you become deeper.

This reminds me of residual connections from CNNs and vision.

snthpy• 2 years ago

Interesting. I hadn't considered that before but makes sense.

Dola Decoding by Contrasting Layers Improves Factuality in Large Language Models

Related Stories