Large models of what? Mistaking engineering achievements for linguistic agency

by Anon84 • 2 years ago

184 156 2 years ago

mnkv• 2 years ago

Good summary of some of the main "theoretical" criticism of LLMs but I feel that it's a bit dated and ignores the recent trend of iterative post-training, especially with human feedback. Major chatbots are no doubt being iteratively refined on the feedback from users i.e. interaction feedback, RLHF, RLAIF. So ChatGPT could fall within the sort of "enactive" perspective on language and definitely goes beyond the issues of static datasets and data completeness.

Sidenote: the authors make a mistake when citing Wittgenstein to find similarity between humans and LLMs. Language modelling on a static dataset is mostly not a language game (see Bender and Koller's section on distributional semantics and caveats on learning meaning from "control codes")

dartos2• 2 years ago

FWIW even more recently, models have been tuned using a method called DPO instead of RLHF.

IIRC DPO doesn’t have human feedback in the loop

valec• 2 years ago

it does. that's what the "direct preference" part of DPO means. you just avoid training an explicit reward model on it like in rlhf and instead directly optimize for log probability of preferred vs dispreferred responses

hackernewds• 2 years ago

DPO most essentially has human feedback, depends on what the preference optimizations are

mistrial9• 2 years ago

oh what a kettle of worms here... Now the mind must consider "repetitive speech under pressure and in formal situations" in contrast and comparison to "limited mechanical ability to produce grammatic sequences of well-known words" .. where is the boundary there?

I am a fan of this paper, warts and all ! (and the paper summary paragraph contained some atrocious grammar btw)

Animats• 2 years ago

Full paper: [1].

Not much new here. The basic criticism is that LLMs are not embodied; they have no interaction with the real world. The same criticism can be applied to most office work.

Useful insight: "We (humans) are always doing more than one thing." This is in the sense of language output having goals for the speaker, not just delivering information. This is related to the problem of LLMs losing the thread of a conversation. Probably the only reasonably new concept in this paper.

Standard rant: "Humans are not brains that exist in a vat..."

"LLMs ... have nothing at stake." Arguable, in that some LLMs are trained using punishment. Which seems to have strong side effects. The undesirable behavior is suppressed, but so is much other behavior. That's rather human-like.

"LLMs Don’t Algospeak". The author means using word choices to get past dumb censorship algorithms. That's probably do-able, if anybody cares.

[1] https://arxiv.org/pdf/2407.08790

ainoobler5• 2 years ago

The optimization process adjusts the weights of a computational graph until the numeric outputs align with some baseline statistics of a large data set. There is no "punishment" or "reward", gradient descent isn't even necessary as there are methods for modifying the weights in other ways and the optimization still converges to a desired distribution which people claim is "intelligent".

The converse is that people are "just" statistical distributions of the signals produced by them but I don't know if there are people who claim they are nothing more than statistical distributions.

I think people are confused because they do not really understand how software and computers work. I'd say they should learn some computability theory to gain some clarity but I doubt they'd listen.

bubblyworld6• 2 years ago

If you really want to phrase it that way, organisms like us are "just" distributions of genes that have been pushed this way and that by natural selection until they converged to something we consider intelligent (humans).

It's pretty clear that these optimisation processes lead to emergent behaviour, both in ML and in the natural sciences. Computability theory isn't really relevant here.

KHRZ• 2 years ago

That's a lot of thinking they've done about LLMs, but how much did they actually try LLMs? I have long threads where ChatGPT refine solutions to coding problems. Their example of losing the thread after printing a tiny list of 10 philosophers seems really outdated. Also it seems LLMs utilize nested contexts as well, for example when it can break it' own rules while telling a story or speaking hypothetically.

tkgally• 2 years ago

For a paper submitted on July 11, 2024, and with several references to other 2024 publications, it is indeed strange that it gives ChatGPT output from April 2023 to demonstrate that “LLMs lose the thread of a conversation with inhuman ease, as outputs are generated in response to prompts rather than a consistent, shared dialogue” (Figure 1). I have had many consistent, shared dialogues with recent versions of ChatGPT and Claude without any loss of conversation thread even after many back-and-forths.

Der_Einzige2• 2 years ago

Most LLM critics (and singularity-is-near influencers) don't actually use the systems enough to have relevant opinions about them. The only really good sources of truth is the chatbot-arena from lmsys and the comment section of r/localllama (I'm quoting Karpathy), both are "wisdom of the crowd" and often the crowd on r/localllama is getting that wisdom by spending hours with one hand on the keyboard and another under their clothes.

GeneralMayhem4• 2 years ago

I am highly skeptical of LLMs as a mechanism to achieve AGI, but I also find this paper fairly unconvincing, bordering on tautological. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed; it's more like it's showing me ways of explaining or illustrating what I already believed.

On embodiment - yes, LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text. The argument feels circular: LLMs are similar to a fake "video game" world because they aren't real people - therefore, it's wrong to think that they could be real people? And the other half of the argument is that because LLMs can only see text, they're missing out on the wider world of non-textual communication; but then, does that mean that human writing is not "real" language? This argument feels especially weak in the face of multi-modal models that are in fact able to "see" and "hear".

The other flavor of argument here is that LLM behavior is empirically non-human - e.g., the argument about not asking for clarification. But that only means that they aren't currently matching humans, not that they couldn't.

Basically all of these arguments feel like they fall down to the strongest counterargument I see proposed by LLM-believers, which is that sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing. If we say that it's impossible to have true language skills without implicitly having a representation of self and environment, and then we see an entity with what appears to be true language skills, we should conclude that that entity must contain within it a representation of self and environment. That argument doesn't rely on any assumptions about the mechanism of representation other than a reliance on physicalism. Looking at it from the other direction, if you assume that all that it means to "be human" is encapsulated in the entropy of a human body, then that concept is necessarily describable with finite entropy. Therefore, by extension, there must be some number of parameters and some model architecture that completely encode that entropy. Questions like whether LLMs are the perfect architecture or whether the number of parameters required is a number that can be practically stored on human-manufacturable media are engineering questions, not philosophical ones: finite problems admit finite solutions, full stop.

Again, that conclusion feels wrong to me... but if I'm being honest with myself, I can't point to why, other than to point at some form of dualism or spirituality as the escape hatch.

abernard17• 2 years ago

LLMs do not have corporeal experience. But it's not obvious that this means that they cannot, a priori, have an "internal" concept of reality, or that it's impossible to gain such an understanding from text.

I would argue it is (obviously) impossible the way the current implementation of models work.

How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperature" have a conceptual model underpinning it? Even theoretically?

Humans and animals have an obvious conceptual understanding of the world. Before we "emit" a word or a sentence, we have an idea of what we're going to say. This is obvious when talking to children, who know something and have a hard time saying it. Clearly, language is not the medium in which they think or develop thoughts, merely an imperfect (and often humorous) expression of it.

Not so with LLMs!! Generative LLMs do not have a prior concept available before they start emitting text. That the "temperature" can chaotically change the output as the tokens proceed just goes to show there is no pre-existing concept to reference. It looks right, and often is right, but generative systems are basically always hallucinating: they do not have any concepts at all. That they are "right" as often as they are is a testament to the power of curve fitting and compression of basis functions in high dimensionality spaces. But JPEGs do the same thing, and I don't believe they have a conceptual understanding of pictures.

GeneralMayhem4• 2 years ago

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

Davidzheng• 2 years ago

It's only because you can essentially put the llms in a simulations that you can have this argument. We can imagine the human brain also in a simulation which we can replay over and over again and adjust various parameters of the physical brain to change the temperature. These sort of arguments can never distinguish between llm and humans.

gwervc• 2 years ago

generative systems are basically always hallucinating: they do not have any concepts at all. That they are "right" as often as they are is a testament to the power of curve fitting and compression of basis functions in high dimensionality spaces

It's refreshing to read someone who "got it". Sad that before my upvote the comment was grayed out.

Any proponent of conceptual or other wishful/magical thinking shoud come with proofs, since it is the hypothesis that diverge from the definition of a LLM.

buu700• 2 years ago

On that point, I would dispute the premise that "it's impossible to have true language skills without implicitly having a representation of self and environment". I don't see any contradiction between the following two ideas:

1. LLMs inherently lack any form of consciousness, subjective experience, emotions, or will

2. A sufficiently advanced LLM with sufficient compute resources would perform on par with human intelligence at any given task, insofar as the task is applicable to LLMs

drdeca3• 2 years ago

I would argue it is (obviously) impossible the way the current implementation of models work.
How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperature" have a conceptual model underpinning it? Even theoretically?

Any probability distribution over strings can theoretically be factored into a product of such a “probability that next token is x given that the text so far is y”. Now, whether a probability distribution over strings can efficiently computed in this form, is another question. But, if we are being so theoretical that we don’t care about the computational cost (as long as it is finite), then the “it is next token prediction” can’t preclude anything which “it produces a probability distribution over strings” doesn’t already preclude.

As for the temperature, given any probability distribution over a discrete set, we can modify it by adding a temperature parameter. Just take the log of the probabilities according to the original probability distribution, scale them all by a factor (the inverse of the temperature, I think. Either that or the temperature, but I think it is the inverse of the temperature.), then exponentiate each of these, and then normalize to produce a probability distribution.

So, the fact that they work by next token prediction, and have a temperature parameter, cannot imply any theoretical limitation that wouldn’t apply to any other way of expressing a probability distribution over strings, as far as discussing probability distributions in the abstract, over strings, rather than talking about computational processes that implement such probability distributions over strings.

But also like, going between P(next token is x | initial string so far is y) and P(the string begins with z) , isn’t that computationally costly? Well, in one direction anyway. Because like, P(next token is x|string so far is y) = P(string begins with yx) / P(string begins with y) .

Though, one might object to P(string starts with y) over P(string is y) ?

fshbbdssbbgdd• 2 years ago

How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperature" have a conceptual model underpinning it? Even theoretically?

Could a creature that simply evolved to survive and reproduce possibly have a conceptual model underpinning it? Model training and evolution are very different processes, but they are both ways of optimizing a physical system. It may be the case that evolution can give rise to intelligence and model training can’t, but we need some argument to prove that.

bubblyworld6• 2 years ago

Transformer models have been shown to spontaneously form internal, predictive models of their input spaces. This is one of the most pervasive misunderstandings about LLMs (and other transformers) around. It is of course also true that the quality of these internal models depends a lot on the kind of task it is trained on. A GPT must be able to reproduce a huge swathe of human output, so the internal models it picks out would be those that are the most useful for that task, and might not include models of common mathematical tasks, for instance, unless they are common in the training set.

Have a look at the OthelloGPT papers (can provide links if you're interested). This is one of the reasons people are so interested in them!

IanCal• 2 years ago

How could a system which produces a single next word based upon a likelihood and and a parameter called a "temperature" have a conceptual model underpinning it? Even theoretically?

You're limiting your view of their capabilities on the output format.

Not so with LLMs!! Generative LLMs do not have a prior concept available before they start emitting text.

How do you establish that? What do you think of othellogpt? That seems to form an internal world model.

That the "temperature" can chaotically change the output as the tokens proceed

Changing the temperature forcibly makes the model pick words it thinks fit worse. Of course it changes the output. It's like an improv game with someone shouting "CHANGE!".

Let's make two tiny changes.

One, let's tell a model to use the format

<innerthought>askjdhas</innerthought> as the voice in their head, and <speak>blah</speak> for the output.

Second, let's remove temperature and keep it at 0 so we're not playing a game where we force them to choose different words.

Now what remains of the argument?

l33tbro• 2 years ago

I've always loved your takes on AI. You should air them here a bit more.

Lerc4• 2 years ago

that sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing.

While sufficiently does a lot of the heavy lifting here, the indistinguishable criteria implicitly means there must be no-way to tell if it is not the real thing. The belief that it is the real thing comes from the intuition that anything that can be everything a person must be, but have that fundamental essence of being a person. I don't think people could really conceive an alternative without resorting to prejudice which they could equally apply to machines or people.

I take the arguments such as in this paper to be instead making the claim that because X cannot be Y you will never be able to make X indistinguishable from Y. It is more a prediction of future failure than a judgment on an existing thing.

I end up looking at some of these complaints from the point of view of my sometimes profession of Game Developer. When I show someone a game in development to playtest they will find a bunch of issues. The vast majority of those issues, not only am I already aware of, but I have a much more detailed perspective of what the problem is and how it might be fixed. I have been seeing the problem, over and over, every day as I work. The problem persists because there are other things to do before fixing the issue, some of which might render the issue redundant anyway.

I feel like a lot of the criticisms of AI are like this they are like the playtesters pointing out issues in the current state where those working on the problems are generally well aware of particular issues and have a variety of solutions in mind that might help.

Clear statements of deficiencies in ability are helpful as a guide to measure future success.

I'm also in the camp that LLM's cannot be an AGI on its own, on the other hand I do think the architecture might be extended to become one. There is an easy out for any criticism to say, "Well, it's not an LLM anymore".

In a way that ends up with a lot of people saying

.The current models cannot do the things we know the current models cannot do

.Future models will not be able to do those things if they are the same as the current ones

.Therefore the things that will be able to do those things will be different

That is true, but hardly enlightening.

et1337• 2 years ago

Future models will not be able to do those things if they are the same as the current ones

I think a lot of people disagree with this. People think if we just keep adding parameters and data, magic will happen. That’s kind of what happened with ChatGPT after all.

skybrian2• 2 years ago

One of the issues here is that future-focused discussions often lead to wild speculation because we don’t know the future. Also, there’s often too much confidence in people’s preferred predictions (skeptical or optimistic) and it would be less heated if we admitted that we don’t know how things will look even a couple of years out, and alternative scenarios are reasonable.

So I think you’re right, it’s not enlightening. Criticism of overconfident predictions won’t be enlightening if you already believe that they’re overconfident and the future is uncertain. Conversations might be more interesting if not so focused on bad arguments of the other side.

But perhaps such criticism is still useful. How else do you deflate excessive hype or skepticism?

brookst2• 2 years ago

sufficiently advanced mimicry is not only indistinguishable from the real thing, but at the limit in fact is the real thing

I am continually surprised at how relevant and pervasive one of Kurt Vonnegut’s major insights is: “we are what we pretend to be, so we must be very careful about what we pretend to be”

Der_Einzige2• 2 years ago

This ideas is older than him by a lot

https://en.wikipedia.org/wiki/Life_imitating_art

Everyone in the "life imitates art, not the other way around" camp (and also neo-platonists/gnostics i.e. https://en.wikipedia.org/wiki/Demiurge ) is getting massively validated by the modern advances in AI right now.

sriku• 2 years ago

The crux of the video game analogy seems to be that when you go close to an object, the resolution starts blurring and the illusion gets broken, and there is a similar thing that happens with LLMs (as of today) as well. This is, so far, reasonable based on daily experience with these models.

The extension of that argument being made in the paper is that a model trained on language tokens spewed by humans is incapable of actually reaching that limit where this illusion will never breakdown in resolution. That also seems reasonable to me. They use the word "languaging" in verb form as opposed to "language" as a noun to express this.

8n4vidtmkvmk• 2 years ago

Why are LLMs incapable of reaching that limit? It's very easy to imagine video games getting to that point. We have all the data to see objects right down to the atomic level, which is plenty more than you'd need for a game. It's mostly a matter of compute. Why then should LLMs breakdown if they can at least mimic the smartest humans? We don't need "resolution" beyond that.

exe342• 2 years ago

. I feel similarly about this as to what I've read of Chalmers - I agree with pretty much all of the conclusions, but I don't feel like the text would convince me of those conclusions if I disagreed;

my limited experience of reading Chalmers is that he doesn't actually present evidence - he goes on a meandering rant and then claims to have proved things that he didn't even cover. it was the most infuriating read of my life, I heavily annotated two chapters and then finally gave up and donated the book.

zoogeny2• 2 years ago

I haven't read any Chalmers so I can't comment on his writing style. I have seen him in several videos on discussion panels and on podcasts.

One thing I appreciate is he often states his premises, or what modern philosophers seem to call "commitments". I wouldn't go so far as to say he uses air-tight logic to reason from these premises/commitments to conclusions - but at the least his reasoning doesn't seem to stray too far from those commitments.

I think it would be fair to argue that not all of his commitments are backed by physical evidence (and perhaps some of them could be argued to go against some physical evidence). And so you are free to reject his commitments and therefore reject his conclusions.

In fact, I think the value of philosophers like Chalmers is less in their specific commitments and conclusions and more in their framing of questions. It can be useful to list out his commitments and find out where you stand on each of them, and then to do your own reasoning using logic to see what conclusions your own set of commitments forces you into.

vouwfietsman• 2 years ago

Isn't any formal "proof" or "reasoning" that shows that something cannot be AGI inherently flawed, because we have a hard time formally describing what AGI is anyway.

Like your argument: embodiment is missing in LLMs, but is it needed for AGI? Nobody knows.

I feel we first have to do a better job defining the basics of intelligence, we can then define what it means to be an AGI, and only then can we prove that something is, or is not, AGI.

It seems that we skipped step 1 because its too hard, and jumped straight to step 3.

GeneralMayhem4• 2 years ago

Yep, this is a big part of it. Intelligence and consciousness are barely understood beyond "I'll know it when I see it", which doesn't work for things you can't see - and in the case of consciousness, most definitions are explicitly based on concepts that are not only invisible but ineffable. And then we have no solid idea whether these things we can't really define, detect, or explain are intrinsically linked to each other or have a causal relationship in either direction. Almost any definition you pick is going to lead to some unsatisfying conclusions vis a vis non-human animals or "obviously not intelligent" forms of machine learning.

It's a real mess.

Vecr• 2 years ago

AIXItl is a formally described AI. Not an AI you'd want, and not an AI you could really build, but it's there.

dullcrisp• 2 years ago

Everyone seems to want to discuss whether there’s some fundamental qualia preventing my toaster from being an AGI, but no one is interested in acknowledging that my toaster isn’t an AGI. Maybe a larger toaster would be an AGI? Or one with more precise toastiness controls? One with more wattage?

plasticeagle2• 2 years ago

There are many finite problems that absolutely do not admit finite solutions. Full stop.

I think the deeper point of the paper is that you simply cannot generate an intelligent entity by just looking at recorded language. You can create a dictionary, and a map - but one must not mistake this map for the territory.

mitthrowaway24• 2 years ago

The human brain is a finite solution, so we already have an existence proof. That means a lot for our confidence in the solvability of this kind of problem.

It is also not universally impossible to reconstruct a function of finite complexity from only samples of its inputs and outputs. It is sometimes possible to draw a map that is an exact replica of the territory.

YeGoblynQueenne12• 2 years ago

> Again, that conclusion feels wrong to me... but if I'm being honest with myself, I can't point to why, other than to point at some form of dualism or spirituality as the escape hatch.

I like how Chomsky deals with it who doesn't have any spirituality at all, the big degenerate materialist:

As far as I can see all of this [he's speaking about the Loebner Prize and the Turing test in general] is entirely pointless. It's like asking how we can determine empirically whether an aeroplane can fly the answer being if it can fool someone into thinking that it's an eagle under some conditions.

https://youtu.be/0hzCOsQJ8Sc?si=MUXpmIwAzcla9lvK&t=2052

(My transcript)

He's right, you know. It should be possible to tell whether something is intelligent just as easily as it is to say that something is flying. If there are endless arguments about it, then it's probably not intelligent (yet). Conversely, if everyone can agree it is intelligent then it probably is.

persnickety6• 2 years ago

I can't disagree more. Or maybe I actually agree.

Because it's not easy to tell whether something is flying. Definitions like that fall apart every time we encounter something out of the ordinary. If you take the criterion of "there's no discussion about it", then you're limiting the definition to that which is familiar, not that which is interesting.

Is an ekranoplan flying? Is an orbiting spaceship flying? Is a hovercraft flying? Is a chicken flapping its wings over a fence flying?

Your criterion would suggest the answer of "no" to any of those cases, even though those cover much of the same use cases as flying, and possibly some new, more interesting ones.

And I don't think an AGI must be limited to the familiar notion of intelligence to be considered an AGI, or, at the very least, to open up avenues that were closed before.

epicfile• 2 years ago

The only thing this paper prove is that folks at Trinity College in Dublin are poor, envious anthropocentric drunkards, ready to throw every argument to defend their crown of creating, without actually understanding the linguistics concepts they use to make their argument.

Salgat• 2 years ago

To me LLMs seem to most closely resemble the regions of the brain used for converting speech to abstract thought and vice-versa, because LLMs are very good at generating natural language and knowing the flow of speech. An LLM is similar to if you took the the Wernicke's and Broca's Areas and stuck a regression between them. The problem is that the regression in the middle is just a brute force of the entire world's knowledge instead of a real thought.

randcraw• 2 years ago

I think the major lessons from the success of LLMs are two: 1) the astonishing power of a largely trivial association engine based only on the semantic categories inferred by word2vec, and 2) that so much of the communication abilities of the human mind require so little rational thought (since LLMs demonstrate essentially none of the skills in Kahneman's and Tversky's System 2 thinking (logic, circumspection, self-correction, reflection, etc).

I guess this also disproves Minsky's 'Society of Mind' conjecture - a large part of human cognition (System 1) does not require the complex interaction of heterogeneous mental components.

zoogeny2• 2 years ago

On embodiment - yes, LLMs do not have corporeal experience.

My own thought on this (as someone who believes embodiment is essential) is to consider the rebuttals to Searle's Chinese Room thought experiment.

For now (and the foreseeable future) humans are the embodiment of LLMs. In some sense, we could be seen as playing the role of a centralized AIs nervous system.

TeMPOraL2• 2 years ago

Rebuttals of Chinese rooms are also rebuttals of embodiment as a requirement! To say the system of person+books speaks Chinese is to say that good enough emulation of a process has all the qualities of the emulated process, and can substitute for it. Embodiment then cannot be essential, because we could emulate it instead.

belter• 2 years ago

"Beyond the Hype: A Realistic Look at Large Language Models" - https://news.ycombinator.com/item?id=41026484

dboreham• 2 years ago

The first stage is denial.

nativeit2• 2 years ago

Well, I suppose that's rather convenient.

nativeit2• 2 years ago

I'm more or less a layperson when it comes to LLMs and this nascent concept of AI, but there's one argument that I keep seeing that I feel like I understand, even without a thorough fluency with the underlying technology. I know that neural nets, and the mechanisms LLMs employ to train and form relational connections, can plausibly be compared to how synapses form signal paths between neurons. I can see how that makes intuitive sense.

I'm struggling to articulate my cognitive dissonance here, but is there any empirical evidence that LLMs, or their underlying machine learning technology, share anything at all with biological consciousness beyond a convenient metaphor for describing "neural networks" using terms borrowed from neuroscience? I don't know that it necessarily follows that just because something was inspired by, or is somehow mimicking, the structure of the brain and its basic elements, that it should necessarily relate to its modeled reality in any literal way, let alone provide a sufficient basis for instantiating a phenomena we frankly know very little about. Not for nothing, but our models naturally cannot replicate any biological functions we do not fully understand. We haven't managed to reproduce biological tissues that are exponentially less complex than the brain, are we really claiming that we're just jumping straight past lab-grown t-bones to intelligent minds?

I'm sure most of the people reading this will have seen Matt Parker's videos where they "teach" matchbooks to win a game against humans. Is anyone suggesting those matchbooks, given infinite time and repetition, would eventually spark emergent consciousness?

The argument would be that that conceptual model is encoded in the intermediate-layer parameters of the model, in a different but analogous way to how it's encoded in the graph and chemical structure of your neurons.

Sorry if I have misinterpreted anyone. I honestly thought all the "neuron" and "synapse" references were handy metaphors to explain otherwise complex computations that resemble this conceptual idea of how our brains work. But it reads a lot like some of the folks in this thread believe it's much more than metaphors, but rather a literal analog.

obirunda3• 2 years ago

I don't think anyone in research actually believes this. Note that the whole idea behind claiming "scaling laws" will infinitely improve these models is a funding strategy rather than a research one. None of these folks think human-like consciousness will "rise" from this effort, even though they veil it to continue the hype-cycle. I guarantee all these firms are desperately looking for architectural breakthroughs, even while they wax poetic about scaling laws, they know there is a bottleneck ahead.

Notice how LeCun is the only researcher being honest about this in a public fashion. Meta is committed to AI already and will at least match the spend of competitors anyway, so he doesn't have as much pressure to try and convince investors that this rabbit whole is deeper.

Don't get me wrong, LLMs are a tremendous improvement on knowledge compression and distillation, but it's still unreliable enough that old school search is likely a superior method nonetheless.

xpe7• 2 years ago

I don’t hold LeCun’s opinions in high regard because of his often hyperbolic statements.

xpe7• 2 years ago

Put aside consciousness or hype or investment. Look at the results; LLMs are well beyond old-school search in many ways. Sure, they are flawed in someways. Previous paradigms for search, were also flawed in their own ways.

Look at the arc of NLP. Large language models fit the pattern. One could even say that their development (next token prediction with a powerful function approximator) is obvious in hindsight.

throwthrowuknow3• 2 years ago

There isn’t really any reason biological neurons should relate to their modelled reality, what does a single cell care about poetry or even simple things like a chair?

bamboozled• 2 years ago

A chair isn't only a chair, it can be a table, a bookshelf, and many others things. The real world is hard.

xpe7• 2 years ago

I find discussions of consciousness even more taxing than religion, free will, or politics.

With very careful discussion, there are some really interesting concepts in play. This paper however does not strike me as worth most people’s time. Especially not regarding consciousness.

kazinator2• 2 years ago

The authors of this paper are just another instance of the AI hype being used by people who have no connection to it, to attract some kind of attention.

"Here is what we think about this current hot topic; please read our stuff and cite generously ..."

Language completeness assumes that a distinct and complete thing such as `a natural language' exists, the essential characteristics of which can be effectively and comprehensively modelled by an LLM

Replace "LLM" by "linguistics". Same thing.

The assumption of data completeness relies on the belief that a language can be quantified and wholly captured by data.

That's all that a baby has, who becomes a native speaker of their surrounding language. Language acquisition does not imply totality of data. Not every native speaker recognizes exactly the same vocabulary and exactly the same set of grammar rules.

IshKebab• 2 years ago

Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure.

But anyway I agree with you. This is just a confused HN comment in paper form.

xpe7• 2 years ago

I personally don’t get much value out of the paper, but it is orders of magnitude more substantive and thoughtful than a median “confused Hacker News comment”.

keybored• 2 years ago

Babies have feedback and interaction with someone speaking to them. Would they learn to speak if you just dumped them in front of a TV and never spoke to them? I'm not sure.

Feedback and interaction is not vital for acquisition for secondary language learning at least according to one theory.

And if that’s good enough for adults it might be good enough for sponge-brain babies.

https://en.wikipedia.org/wiki/Input_hypothesis

JohnKemeny• 2 years ago

They are two researchers/assistant professors working with cognitive science, psychology, and trustworthy AI. The paper is peer reviewed and has been accepted for publication in the Journal of Language Sciences.

You should publish your critique of their research in that same journal.

P.s. if you find any grave mistakes, you can contact the editor in chief, who happens to be a linguist.

kazinator2• 2 years ago

You should publish your critique of their research in that same journal.

No thanks; that would be at least twice removed from Making Stuff.

(Once removed is writing about Making Stuff.)

lucianbr2• 2 years ago

An appeal to authority if ever there was one.

Their critique is written here, in plain english. Any fault with it you can just mention. The "I won't read your comment unless you get X journal to publish it" seems really counterproductive. Presumably even the great Journal of Language Sciences is not above making mistakes or publishing things that are not perfect.

mquander• 2 years ago

The "efficient journal hypothesis" -- if something is written in a paper in a journal, then it's impossible for anyone to know any better, since if they knew better, they would already have published the correction in a journal.

xpe7• 2 years ago

Please argue on the merits and substance. I’m less interested in speculation on the authors’ motivations.

xpe7• 2 years ago

The parent comment I responded to is speculative and does not argue on the merits. We can do better here.

Are there people who ride the hype wave of AI? Sure.

But how can you tell from where you sit? How do you come to such a judgment? Are you being thoughtful and rational?

Have you considered an alternative explanation? I think the odds are much greater that the authors’ academic roots/training is at odds with what you think is productive. (This is what I think, BTW. I found the paper to be a waste of my time. Perhaps others can get value from it?)

But I don’t pretend to know the authors’ motivations, nor will I cast aspersions on them.

When one casts shade on a person like the comment above did, one invites and deserves this level of criticism.

amne• 2 years ago

tl:dr; we're duck-typing LLMs as AGI

Simon_ORourke• 2 years ago

Where I work, there's a somewhat haphazardly divided org structure, where my team has some responsibility to answer the executives demands for "use AI to help our core business". So we applied off-the-shelf models to extract structured context from mostly unstructured text - effectively a data engineering job - and thereby support analytics and create more dashboards for the execs to mull over.

Another team, with a similar role in a different part of the org has jumped (feet first) into optimizing large language models to turn them into agents, without consulting the business about whether they need such things. RAG, LoRA and all this optimization is well and good, but this engineering focus has found no actual application, expect wasting several million bucks hiring staff to do something nobody wants.

rramadass• 2 years ago

See also Beyond the Hype: A Realistic Look at Large Language Models • Jodie Burchell • GOTO 2024 - https://www.youtube.com/watch?v=Pv0cfsastFs

flimflamm• 2 years ago

How would the authors consider a paralyzed individual who can only move their eyes since birth? That person can learn the same concepts as other humans and communicate as richly (using only their eyes) as other humans. Clearly, the paper is viewing the problem very narrowly.

fairthomas• 2 years ago

...a paralyzed individual who can only move their eyes since birth...

I don't think such an individual is possible.

throwthrowuknow3• 2 years ago

I didn’t want to Google it for you because it always makes me sad but things like spina bifida and moebius syndrome exist. Not everyone gets to begin life healthy.

throwthrowuknow3• 2 years ago

“Enactivism” really? I wonder if these complaints will continue as LLMs see wider adoption, the old first they ignore you, then they ridicule you, then they fight you… trope that is halfways accurate. Any field that focuses on building theories on top of theories is in for a bad time.

https://en.m.wikipedia.org/wiki/Enactivism

beepbooptheory2• 2 years ago

What is the thing the LLMs are fighting for?

beepbooptheory2• 2 years ago

There is a lot of frustration here over what appears to be essentially this claim:

...we argue that it is possible to offer generous interpretations of some aspects of LLM engineering to find parallels with human language learning. However, in the majority of key aspects of language learning and use, most specifically in the various kinds of linguistic agency exhibited by human beings, these small apparent comparisons do little to balance what are much more deep-rooted contrasts.

Now, why is this so hard to stomach? This is the argument of this paper. To feel like this extremely general claim is something you have to argue against means you believe in a fundamental similarity between what our linguistic agency and the model. But is embodied human agency something that you really need the LLMs to have right now? Why? What are the stakes here? The ones actually related to the argument at hand?

This ultimately not that strong of a claim! To the point that its almost vacuous... Of course the LLM will never learn the stove is "hot" like you did when you were a curious child. How can this still be too much to admit for someone? What is lost?

It makes me feel little crazy here that people constantly jump over the text at hand whenever something gets a little too philosophical, and the arguments become long pseudo-theories that aren't relevant to argument.

Royshiloh• 2 years ago

Why assume you "know" what language is? Like there is a study backed insight on the ultimate definition of language? it's the same as saying "oh, it's not 'a,b,c' its 'x,y,z'", which makes you as dogmatic as the one you critique. This is absurd.

Large models of what? Mistaking engineering achievements for linguistic agency

Related Stories