Does the success of LLM support Wittgenstein's position that "meaning is use"?

https://philosophy.stackexchange.com/questions/112021/does-the-success-of-ai-large-language-models-support-wittgensteins-position-t

by IdealeZahlen • 2 years ago

37 17 2 years ago

philosophy.stackexchange.com

crystal_revenge• 2 years ago

Very refreshing to see some philosophical discussions around LLMs! As someone interested in both philosophy and LLMs it's fascinating how many meta-physical questions dip their toes into the practical without many people realizing it.

For example: the premise that, with enough data and compute resources, LLMs will simply stop hallucinating. This implies there is some latent space of "Truth", essentially the empirical realization of Platonic idealism.

What's funny is that I've often seen people argue aggressively in favor that LLMs will stop hallucinating who I'm pretty sure would also strongly reject Platonic idealism, leading to a sort of implied contradiction in their reasoning.

idle_zealot• 2 years ago

Consider that what people often mean by "stop hallucinating" is more like "stop making errors that a human wouldn't." I don't think most people believe that a powerful enough LLM would be capable of 100% accuracy in all tasks.

neilk• 2 years ago

Not really. Wittgenstein wasn’t making the radical claim that words just float around in utterances, with no connection to ideas in a human mind.

He saying that language doesn’t reduce down to simple “atomic” forms, as he had previously thought. Much of speech is embedded in context in the way we live. You couldn’t take “I CAN HAZ CHEEZBURGER?” and reduce it down to something just by looking at the words and their relation to each other. The person using this statement is certainly not saying “For all elements of the set cheeseburger, there exists in the set of all possible futures one element which is proper to the set of elements identical to myself”. They are invoking a context, a mode of thinking, even a shared memory. This is why the meme is hilarious in some contexts, but would be utterly unintelligible in others.

The existence of LLMs does not provide support for Wittgenstein’s theories, which are more about observing humans - it’s sometimes described as an “anthropological” theory of language. The fact that it’s an impersonal process doesn’t add much. Humans can find random poetry assembled with fridge magnets meaningful too, but that says little about the nature of language.

PS Also, it’s well known that embeddings do seem to have a kind of “conceptual” relationship to each other. Famously some people find that the vector difference of king->queen will usually be related to man->woman. So, in some bizarre high dimensional way they do sort of have concepts. And given enough data they also have contexts. It’s extremely unclear if this is analogous to how humans actually think, but maybe LLMs have some facilities that are similarly powerful.

repelsteeltje• 2 years ago

And one of the areas Wittgenstein discussed wrt. speech act are anaphores, deixis, etc. The meaning of "now", "that" or "me" - anchoring words in reality.

In this context, even multi modal LLMs (while impressive feats) are child's play.

karaterobot4• 2 years ago

I'm someone who insists that usage is only one factor, even if it's an important one. I hold on to the idea that authority and historical meaning should not be completely thrown out with the bathwater.

I've had arguments with people who hold strong opposing positions, and I think that's probably a more normal one to hold than mine, at least among my set.

But recently, one of the people with whom I've had this argument posed the "are hotdogs a sandwich?" question. I said no, he said yes.

At that moment, the exact question posed in this stackexchange question occurred to me, so I went to ChatGPT and asked it to give me a 10x10 grid of pictures of sandwiches. The prompt was `A 10x10 grid of illustrations. In each cell of the grid is a picture of a sandwich.`

Of course it didn't give me a 10x10 grid, because it can't count like that. But it did give me a series of grids of illustrations. I asked the same prompt again, over and over. It took over 200 total images of sandwiches for it to show me a picture that included a hotdog. The hotdog was sticking out from between two pieces of rye bread. It also produced a picture of a whole lobster lying on top of a pita, and a block of sushi wrapped in nori, and an ordinary bowl with a salad in it.

My (trollish) position in that thread became that there is no better authority on common use than ChatGPT, whose entire purpose is to distill it and recapitulate it. So, if meaning is use, hotdogs are definitely not sandwiches—at least they are less clearly so than a bowl of salad is.

famouswaffles3• 2 years ago

At that moment, the exact question posed in this stackexchange question occurred to me, so I went to ChatGPT and asked it to give me a 10x10 grid of pictures of sandwiches. The prompt was `A 10x10 grid of illustrations. In each cell of the grid is a picture of a sandwich.` Of course it didn't give me a 10x10 grid, because it can't count like that.

All ChatGPT is doing is sending a prompt to Dalle-3...

karaterobot4• 2 years ago

Explain the distinction and relevance for me please!

cassianoleal• 2 years ago

I think this is partly due to the usage of the very word "hotdog".

In Brazil, cachorro-quente (a literal translation of hotdog to Portuguese) is a sausage in bread - with various other ingredients thrown in depending on many factors, including location (in São Paulo, it's common to add mashed potato; in Rio Grande do Sul, it's common to add "batata-palha" - crunchy potato sticks), etc. Some add sweetcorn. There's usually lettuce, some times other vegetables.

That, in my mind, is undeniably a sandwich.

In the US, as far as I can tell, the sausage alone can be called a hotdog. In Young Sheldon, they frequently have "hotdog spaghetti", which seems to be pasta with sausage cut in chunks.

The sausage alone is not, by any definition, a sandwich.

I wonder if it's at least partially due to this usage that ChatGPT has a tendency to not assume "sandwich" from "hotdog".

taeric2• 2 years ago

But, by this measure, you could probably conclude that a Joshua Tree is not a tree? How many times would you have to ask for a grid of trees before it produced one? (And, obviously, pick any other non-widespread tree at your leisure.)

karaterobot4• 2 years ago

Good point. But, if a Joshua Tree is a rare and not particularly prominent topic of conversation, I wouldn't necessarily expect it to come up when talking about trees. On the other hand, hotdogs are not rare, they're popular and widespread. They are only marginal when talking about sandwiches, which is my point.

SoftTalker• 2 years ago

The question is, which is to be master — that’s all.

mordymoop• 2 years ago

It is unclear to me whether an empirical fact about cognitive systems can even in principle support or refute an assertion of the abstract form “<ambiguous concept> is <other ambiguous concept>.”

One could also point out the the underlying assumption of the question seems to rest on LLMs as representing “just statistics on language”, which is only a correct assertion insofar as babies learn language by just doing statistics on language. I am at this point not even sure what people think they mean when they say this, what they think these “statistics” are, and why they think calling the system “statistics” instead of “cognitive algorithms” does any philosophical work.

In other words, the OP seems to rest on the framing that LLMs are “just” doing “statistical analysis” that this is some kind of meaningful distinction, as if the argument would change if LLMs were doing some other kind of analysis.

I can’t escape the sense that I am being overly generous and that the OP simply has no idea how transformers or even neural networks work, but feels very sure that it must be some kind of parlor trick, and thus presumes that it is so.

famouswaffles3• 2 years ago

Yes statistics has basically become magic now.

This seems to be a general overview.

A - Task x requires understanding

B - GPT-N performs Task x

C - Obviously, GPT-N doesn't understand

D - Task x doesn't require understanding

But see, C is just an unfounded assertion. If you question someone on C, they will often say something like, "It's just statistics and that's why it's still possible for B to happen". But "statistics" never does any real work here. Never provides any meaningful distinction. If you replace "statistics" with "magic" in the sentence, nothing actually changes because all statistics does in these kinds of conversations is provide a convenient boogeyman.

A - A force is required to lift a ball

B - I see Human-N lifting a ball

C - Obviously, Human-N cannot produce forces

D - Forces are not required to lift a ball

Well sir, why are you so sure Human-N cannot produce forces? How is he/she lifting the ball ? Well Of course Human-N is just using s̶t̶a̶t̶i̶s̶t̶i̶c̶s̶ magic

343rwerfd• 2 years ago

Lots and lots words flow about this. For me, it is very simple. LLMs do a complex process, quite analog to human thinking/understanding/speaking/writing, but they're doing all those - most probably - in an alternative way to what humans do in their brains.

Directly comparing LLM's outputs with human output is like comparing a F22 flying with an eagle flying. Both fly obviously, but using entirely different processes to do so (different requirements, capabilities, despite the simplest similarity of both systems - the eagle and the F22 - at "doing fly").

You don't automatically say "an eagle is more capable at flying than a F22, because it flies with very little energy requirements while deploying quite better, reliable take-off / landing capabilities".

You actually don't usually go comparing these systems just because both can fly.

but many out there are pulling their hair trying to compare side by side the obvious mathematical systems that LLMs are to - most probably again - the completely different in nature systems that humans are.

Does the success of LLM support Wittgenstein's position that "meaning is use"?

Related Stories