What if AGI is not coming?

https://www.mindprison.cc/p/what-if-agi-is-not-coming

by 13years • 2 years ago

69 181 2 years ago

mindprison.cc

necovek5• 2 years ago

This does not really consider the "what-if" in the title, but mostly puts out the arguments for why is it not coming.

So a bit of a cop-out not wanting to say it outright :)

cykros3• 2 years ago

They know it's not coming, but don't mind seeing their 401(k) go up for awhile.

Why tell the children Santa's not real, when he's the reason they're being so good?

adastra222• 2 years ago

AGI arrived in 2017.

colordrops• 2 years ago

what happened in 2017?

motoxpro• 2 years ago

I think they are talking about the transformer paper?

adastra222• 2 years ago

Transformers. AGI means Artificial General Intelligence. The transformer architecture enables the transfer of knowledge to new domains in arbitrary ways, allowing for the solving of arbitrary problem domains.

jimmcslim• 2 years ago

What if intelligence is an product of consciousness, and consciousness is an product of something that can never have a physical definition and is always ethereal... i.e. a "soul".

If we can achieve AGI simply through more and more computation, no matter how novel it is, its ultimately ifs, loops and arithmetic... then surely the human experience is ultimately just a 'wet LLM' (or whatever we end up calling the machine learning technology behind AGI).

laerus• 2 years ago

Unrelated but do you believe animals have a soul?

zhugeyangyang2• 2 years ago

I think it's just a bubble, containing some internal context

akasakahakada13• 2 years ago

Uhh again the "only human has soul" argument. What means by soul, please define this word, philosopher. Does bird know language grammars indicate that soul exist? Does fish can do arithmetic up to 5 show that their soul exist? Does elephant do funeral to their dead fellow show that there is a soul? Can this humancentric ideology just go away?

err4nt2• 2 years ago

Like humans, they have physical bodies and the breath of life in them. For the sake of what this person was talking about it seems like it would apply to their intelligence as well.

baq2• 2 years ago

What’s the difference between real consciousness and simulated in silico consciousness? If the fake consciousness is good enough there shouldn’t be any.

Is a soul made out of matrix multiplications and dot products worse than a soul made of neurons?

akasakahakada13• 2 years ago

This is the kind of question that every NonSTEM people would struggle at. Given function F and G. For all input x, resulting F(x) = G(x), this indicate that obviously F and G is the same thing whether inside or outside.

exe348• 2 years ago

I like to call it carbon chauvinism. There are people who believe that no matter how high the simulation fidelity, no matter the level you simulate the neurons, you can't simulate consciousness. I think they can be safely ignored until after we've tried.

injidup• 2 years ago

Given that Harry Potter, Jedi Knights and Souls are fantasy items from literature your point with regards to testable silicon based intelligence is what exaclty?

err4nt2• 2 years ago

This is kind of where I've been thinking lately. If I can speak of human bodies like computer hardware, there is a physical limit to our hardware and how much data it can process, and yet intelligence seems to be something beyond the hardware. Something that the living 'you' brings to your hardware.

exitb2• 2 years ago

We already have machines that are generally intelligent and require as much energy as a few light bulbs. Why wouldn’t it be eventually possible to replicate them in silicon?

aumerle• 2 years ago

"eventually" is doing a lot of heavy lifting here.

vbezhenar• 2 years ago

There are plenty of animals with brain. We weren’t able to use them for their cognitive abilities so far. So even if we would replicate the hardware, software might prove challenging.

2muchcoffeeman• 2 years ago

This sort of thinking might be part of the problem. Why are we analogous to software running on hardware?

Phiwise_• 2 years ago

Over a hundred years ago, Babbage might have said:

We already have machines that are generally intelligent and require as much energy as a few coalgas lights. Why wouldn’t it be eventually possible to replicate them in brass and steam?

My understanding of the argument of this article is that the conceptual design that replicates intelligence is what the industry has failed to generate today. Simply stating that it might be possible to create is failing to engage; the massive increase in compute power from Babbage to McCarthy didn't give us AGI, because they didn't figure out the right design to reason about anything in even a hundred times a human's energy consumption. If from McCarthy to today we still haven't actually found the proper recipe it just might be worth considering the point the article's making in spite of our other advancements since then.

omnicognate• 2 years ago

Because they aren't machines, we don't understand how they work and it's very much an open question whether we ever will. OpenWorm tried and failed the get the fully documented behaviours of the C. Elegans nematode (for which we know the complete connectome and the full lineage of every one of the 959 cells of its body) to emerge from simulations of its 302 neurons. We don't understand how an individual neuron works. We don't even understand what an individual neuron really does and the brain contains 100 billion of them. The hubris involved in looking at biological nervous systems and thinking "how hard can it be? Give me a big data centre and 5 years!" is staggering.

card_zero2• 2 years ago

Everybody said the exact same thing while I was busy writing it out. Neat.

exitb2• 2 years ago

I’m specifically speaking out against the bit data center approach and only saying that it’s demonstrably possible to do general intelligence on a small, low power machine.

Probably shouldn’t have mentioned the silicon though.

card_zero2• 2 years ago

These meat machines aren't necessarily computers, that may be only our best available metaphor (previous metaphors for the brain have I think included telephone exchanges, steam engines and plumbing), and although in principle silicon can simulate anything, in practice not so much: there have been various projects to simulate C. elegans, such as OpenWorm, and they fail because the worm depends on the physics of its environment (possibly including the environment inside the worm) and it's all more subtle than just its nervous system. So yeah, it's reasonable that it might eventually be possible to replicate the human brain in something, once we have the slightest clue how and what.

fmbb• 2 years ago

Why wouldn’t it be eventually possible to replicate them in silicon?

I mean if you simulate all the chemical processes in some being then yeah probably you get close (assuming you get everything right), let’s assume there is no unknown in how the atoms in our bodies interact.

Sounds like an expensive project.

If you are _not_ talking about a perfect emulation/simulation of such a machine, I will pose your question back to you: why _would_ it be possible to do something we don’t know what it is? Seems rather contrived to say “we can do this thing we don’t know what it is”.

az09mugen• 2 years ago

That's exactly this 'generally intelligent' that confuses minds. People believe chat gpt has a consciousness but it does not at all. It's just a model of statistics. The first step toward AGI first of all is consciousness on silicon.

And with the current models, the only thing I see is : "let's add more neurons, give it more data to train, and let's hope a mind will come out of these neurons". It helps understand the human brain, and help humanity and all, but I'm sure we are missing some 'theorical ingredients' to get a recipe for a full working AGI.

skybrian4• 2 years ago

The future is not known to us. But given how inefficient machine learning seems to be, algorithmic efficiency improvements may keep the scaling going for a while? Maybe that's not a "major breakthrough" but it's improvement nonetheless.

It's also going to take a while to learn to use the new toys we already have.

rented_mule• 2 years ago

I'd love to see more algorithmic efficiency in ML. But hasn't ML been going in the other direction for at least a decade now? It seems to me that it's been going aggressively towards brute force algorithms. Specifically, figuring out how to do as much matrix multiplication as possible.

skybrian4• 2 years ago

Among other things, there seems to be progress in fixing attention to scale better.

czl2• 2 years ago

given how inefficient machine learning seems to be

We emulate neurons mathematically but it is possible to build efficient analog circuits that emulate them physically.

I doubt any biological system can ever learn all the information gpt4 has learned. The gpt4 learning may not have been power efficient but neither were the first airplanes compared to birds yet today biological flight is rather limited compared to flight that uses technology.

Ever listen to Geoffrey Hinton speak about back propagation vs what biological systems use? Do you think he is wrong about this?

imtringued5• 2 years ago

He isn't wrong, but with current GPU hardware back propagation is actually the superior algorithm compared to his latest forward forward algorithm. Swapping the training algorithm doesn't get around the fact that you need to perform lots and lots of forward passes and those need an insane amount of memory bandwidth. Back propagation isn't a significant bottleneck in computers and also not in terms of memory capacity. So the only benefit of the biologically plausible forward forward algorithm is that you could run it on a digital or analog NPU, but with Ryzen AI coming in Strix Point, every high end laptop is going to have 50 TOPS of AI compute and 200GB/s of memory bandwidth. Nothing except large datacenter GPUs or a hypothetical 5090 with 32 GB would be competitive against that anytime soon. Anything smaller and you will have to buy several 3090 on eBay for like $900 a pop. Analog circuits are far off for now.

namaria9• 2 years ago

I doubt any biological system can ever learn all the information gpt4 has learned

The fact that human brains can become competent at language with training datasets with a size of a tiny fraction of common crawl points to how much more efficient they are

hatenberg• 2 years ago

WHat a strange piece of writing.

"Planes and Cars today fundamentally use the same technology we had for almost decades, henceforth ...."

The real question to ask is "does AGI matter"

maxbaines• 2 years ago

Yes I found it an odd piece also.

aorloff• 2 years ago

If with each step we are halfway closer to the goal of AGI, how long before we get there ?

mirekrusin• 2 years ago

…assuming improvements are halfway or less - which is not what we’re seeing.

beambot• 2 years ago

Ah, Zeno's paradox. Clearly the turtle will win.

chubot• 2 years ago

Predictions aren't worth much without a bet, but I think the tech will plateau in the next decade, for several years or more, just like it has in the past

One main reason is that I think people underestimate how much work OUR brains are doing when we interact with LLMs. It seems like the initial "wow" has worn off for many people, but definitely not everybody.

For coding, people will get stuck in loops, trying to get LLMs to modify LLM-generated code

And I think the market will cool down, which seems inevitable considering Nvidia's stock price (I'm a shareholder), and the fact that they seem to be the only ones really making money

If you compare Google after 8 years (2004) to OpenAI after 8 years (2023), the business is uh very different

diarrhea2• 2 years ago

I feel like the space has already plateaued. Lots of improvements up to GPT4 were genuine milestones, but that’s now a year ago and everything since was marginal.

I’m not invested in any sense in the space. I’m actually more frequently turning off Copilot in VSCode recently. I’d like to see further breakthroughs as much as anyone, but am not holding my breath. In fact, shorting NVIDIA seems like one of the better ideas currently.

seanmcdirmid• 2 years ago

But couldn’t you have said that 5 years ago and 10 years ago? I’d give it 3-4 years to actually call it.

But if you believe strongly, shorting AI-enhanced stocks is a great way to capitalize on your prediction. You could also use the short as a hedge for your expectations (either way you win something).

namaria9• 2 years ago

There's a real chance of rapid decline as the advance we've seen on GPT3 was largely due to openAI being able to efficiently train it on common crawl but now this data body is getting poisoned by automatically generated content.

imtringued5• 2 years ago

The problem with LLMs for code editing is that they generate new tokens, instead of performing in place context editing (IPCE). It probably shouldn't be that difficult to just add a final layer that is trained to perform IPCE. Then you would get a lot of the benefits of MemGPT without an external tool. The LLM could go back and only rewrite a handful of lines.

bottlepalm• 2 years ago

I think we've way over done the 'general intelligence' part of AI already, that is already 'super general intelligence'.

What's lacking is agency/autonomy. I have a bad feeling even 'general autonomy' will take a fraction of the power we're already using which means 'super autonomy'... is probably already possible.

Which means ASI soonish.. which leads to uncontrolled ASI either deliberately or accidently.. which means.. well it's out of our hands at that point. Anything can happen.

topbanana• 2 years ago

One thing's for sure, there are now a lot more people looking to make it happen

anonzzzies• 2 years ago

It’s fine if I doesn’t ; current LLMs are already very helpful; we need them faster, smaller and eating less resources. If not AGI, let’s run 50 personal assistants on my phone.

dagmx4• 2 years ago

I think this is where people will be disappointed when “AI” is brought to mass consumer.

The level of results does not scale well down to mass consumer hardware.

And yes I know people can buy an NVIDIA GPU and run these models, but the phone like you said is the most common computer and where this will be hardest to scale too.

It’s why I’m bearish on AI, and I think the pop will be due to being unable to scale down sufficiently

klyrs• 2 years ago

The level of results does not scale well down to mass consumer hardware.

For now. I'm pretty bearish on all things "AI" but of all things one can say about the future, today's hardware is yesterday's news. And in this case, I'd say the same goes for algorithms.

czl2• 2 years ago

With time mass consumer hardware scales up. You expect that trend to stop?

tavavex2• 2 years ago

The level of results does not scale well down to mass consumer hardware.

I think that it does, though. A few years ago, running any remotely complex (and decent) generative AI on anything that wasn't a large compute server was out of the question. Nowadays, you can run a very respectable image or text generation algorithm on a middle-of-the-road gaming PC. Local models may not always be as good as the enormous things that companies run, but they're putting up a fight.

This isn't founded on research, but given how much people were able to scale all of it down, I feel like there's going to be more of this "fat" to trim - data that takes up a lot of space but isn't very important to the final result. Add onto that the constant improvement of hardware, and the lines are going to intersect eventually.

imtringued5• 2 years ago

Strix Point isn't even out yet. Sure that is a laptop form factor and you won't be able to run those 130b models because of RAM constraints, but things like mixtral 8x7b will easily run at 20 tokens per second locally on your laptop within this year.

aeturnum• 2 years ago

I think "LLMs are using well-studied modeling techniques with overwhelming resource investment" is the most fundamental critique and why I've been skeptical of the future of this wave. That's not to say we won't (and haven't already) gotten useful tools! There's obviously a lot to do with human language interfaces and complex analysis. I'm just skeptical a whole new level is just around the corner.

raincole• 2 years ago

I honestly don't see what the problem is. One can say the internet is just to "connect machines with wires and have a set of protocols allowing them to communicate". It's true, but the magic happens when simple ideas get scaled.

namaria9• 2 years ago

When you have to throw billions of dollars worth of compute at a problem to brute force it, you're not exactly 'scaling it' as much as scaling your costs for diminishing returns.

akasakahakada13• 2 years ago

As long as keep philosophers keep shifting the definition of AGI, that should never come to us.

TuringTest• 2 years ago

Wow, there is a definition?

I'd love to hear it ;-)

akasakahakada13• 2 years ago

ironic name

tolleydbg• 2 years ago

This isn't what is happening though. Philosophers keep poking holes in AGI arguments, previously Strong AI, and techbros keep using a new term, each more ambiguous than the last. The hope, it seems, is to use this ambiguity to prevent pointed criticism that would prevent investment and adoption.

janalsncm3• 2 years ago

One of the things I wonder about is whether “intelligence” can be linearly scaled or if it’s just a way of solving an optimization problem. In other words, humans have come pretty close to the peak of Mt. Smarts and therefore being 1000x as intelligent is more like the difference between 1 meter from the peak and a millimeter from the top. You’re both basically there.

In other words, maybe humans have basically solved the optimization problem for the environment we live in. At this point the only thing to compete on is speed and cost.

blfr• 2 years ago

You don't see that many, or any really, von Neumanns walking around so there's probably still significant room to improve with all the benefits of having intelligence neatly packaged in a computer.

kurthr• 2 years ago

Even if all the computers can do is ask the right questions and it takes a big research project to figure it out, that would be an improvement in productivity.

I actually think it will come from the other direction. That people will get better at asking questions, because there is an automated tool that will build systems to answer larger problems than a single person could quickly answer.

a_wild_dandan• 2 years ago

Yeah, imagine spinning up 100 von Neumanns to attack a problem. They can all instantly share their thoughts & new skills, coordinate, choose new exploration directions, and spend decades developing new tools -- all within moments after pressing 'Enter'.

Even if our AI systems have only a minute fraction of von Neumann's intellect, we still have no idea what tomorrow will be like. I'm terrified and excited.

throwaway48r7r7• 2 years ago

the environment we live in

This has changed drastically and thus our definition of smart has too.

__MatrixMan__2• 2 years ago

I don't think there is such a thing as general intelligence, there are only capabilities. What we call "general" intelligence is really just the set of capabilities that a human has, because we're self-centered.

If we had more intelligences around to compare with I think we'd find that some are "more intelligent" in that they have all of our capabilities, plus some. And that others are "less intelligent" in that we have all of the capabilities that they have, plus some. And then there would be the "differently intelligent" which have at least one capability that we don't and which lack at least one capability that we have.

Under this lens, I don't know if there's much utility in fine grained comparisons of intelligence re meters and millimeters. The space is discrete: subsets, not metrics.

I don't know if you could ever prove something like this (or maybe we just lack that capability). It seems more like an axiom-selecting notion than something to be argued. Anyhow, it's what my gut says.

sinuhe693• 2 years ago

I think it’s an interesting thought. But for the sake of it, can you name/imagine some examples of problems/forms of problems we as human can not solve? Or can not solve efficiently?

If you can find none, is it not the proof that our intelligence is general?

danaris2• 2 years ago

...Or possibly it's the other way around: to get 1000x as smart, you need to put in 1e1000 times the effort needed to get to human-like intelligence.

gorgoiler3• 2 years ago

In life I tend to encounter two common patterns of intelligent people: those who had a good education and those who did not. I worry that when AGI comes it is going to be able to do all the things the smooth fast taking wily folks can do, and none of the things the educated folks can do, and we’ll accelerate not a slide into the singularity but a slide into inane banality.

How do you provoke a model into being wacky, challenging, and innovative?

throwaway48r7r7• 2 years ago

This is exactly what's happening.

bjornsing2• 2 years ago

How do you provoke a model into being wacky, challenging, and innovative?

Are those the typical qualities of the educated…?

gorgoiler3• 2 years ago

I can’t put my finger on it, but I think the intellectually challenging part is at the heart of the matter. A wily person can talk their way through a debate by saying things that sound compelling. An educated person has more ability to reason: they can challenge you when they know they are right, and explain why you are wrong.

It’s the difference between bluffing and sincerity, or dishonesty and truthfulness. Current LLMs are confident liars.

throwaway2037• 2 years ago

    > we’ll accelerate not a slide into the singularity but a slide into inane banality

Half joking response: Have you looked at all the SEO garbage than any Google search produces these days? We are already in the great age of "inane banality".

imtringued5• 2 years ago

I told it to be unfriendly and use black humour. It certainly is wacky, especially with that TTS voice. Running this locally made me realize that OpenAI's ChatGPT is way too uptight and the personality is way too bland. The LLM can already whip out "smooth talk" faster than me.

tjansen• 2 years ago

At the very least, it knows solved problems and repeat the solution. That's better than pretty much all humans, since humans can't know all known solutions.

rvz• 2 years ago

What if 'AGI' was another over-promised scam to sell stochastic parrots marketed as "intelligence" for a product that not even its creators can even understand when it goes wrong badly?

"Oh don't worry, AGI is coming soon and we'll solve that later" - AI founders

Yet they don't even know how long that is since no-one knows or it never happens. Mistakes in AI are costly and are very expensive.

What if their startup fails before the time arrives because they still cannot make any money and need to constantly raise VC money every week or quarter?

Again, there will only be 90% - 95% of these 'AI' companies that will fail with the 5% to 10% still around including the incumbents.

throwaway48r7r7• 2 years ago

This undersells the fact that to a not insignificant degree humans are stochastic parrots too.

danaris2• 2 years ago

This argument is, and always has been, utter bullshit.

All humans have the capacity to genuinely learn, create, and think, regardless of how their output appears to you in some subset of interactions with them.

"Some humans sometimes have trouble with critical thinking, or just regurgitate previously-memorized facts" is not in any way equivalent to "LLMs, by their fundamental nature, only have the capacity to produce various recombinations of their training data."

Arthanos• 2 years ago

"no major AI technology breakthroughs in decades.everything we are seeing is larger compute scaling." This is false. Everything from the transformer to advancements in state space models have been foundational breakthroughs

dvt2• 2 years ago

I beg to differ. Transformers are purely an optimization. It’s not exactly right to call everything “compute scaling” but we are still, at the end of the day, fitting polynomials.

And frankly, that’s probably not what our brains are doing.

mitthrowaway23• 2 years ago

And frankly, that’s probably not what our brains are doing.

I think that it is! It's much more likely to me that our brains are doing something big and simple than small and complicated. That's the way that nature tends to work. Fitting low-order million-dimensional polynomials would meet that description.

f6v2• 2 years ago

It’s difficult to comprehend impact of something that’s widely adopted. Like, batch normalization alone was probably mind blowing when it came out. Yet it seems so simple and self-explanatory now.

bjornsing2• 2 years ago

It is a bit weird though… I mean, you could just as well say that the last breakthrough in all of computing was the transistor (Shockley 1946) – all we’ve been doing ever since is just “scaling” and combining them in new ways.

chii• 2 years ago

People who say things like "major breakthroughs" often imagine a cliff/steep rise. The reality is that most "breakthroughs" are small, incremental, almost invisible steps in all aspects of the field, and potentially even in fields that are only tangentially related.

Then, one day, they hear about it in the news, because there's now some hype, or some event that makes it newsworthy. This makes it feel like the breakthrough was instantaneous or steep, but in fact has been in the making for decades or even more.

I have no doubt AGI is coming, but it will be gradual and slow. It will be the accumulation of more advances in everything, including hardware, as well as software. It might even include economic changes.

akasakahakada13• 2 years ago

From nothing to ChatGPT opened to the public and billions students' lifes rely on that in the next month, isn't this the biggest step in the tech history?

peter_retief5• 2 years ago

We cannot create life on the simplest scale, there have been experiments with the creation of life in the Miller experiment have only produced so called building blocks, amino acids. However we are unable to create life in dead creatures that have all the building blocks in place.

What is happening is the belief that the laws of thermodynamics are probabilistic, like a law that can be broken. Laws like gravity and thermodynamics are deterministic and the hubris of those who make claims of real intelligence in machines we create are going to be as disappointed as those who design perpetual motion machines.

Vecr6• 2 years ago

I don't really understand, if physicist is right and deterministic we already have thinking machines. They're called brains. As far as anyone can tell, thermodynamics works fine for them.

peter_retief5• 2 years ago

Life cannot be created even on the simplest form let alone a brain.

zone411• 2 years ago

I've just created a new benchmark to see how top LLMs do on NYT Connections (https://www.nytimes.com/games/connections). 267 puzzles, 3 prompts for each, uppercase and lowercase.

GPT-4 Turbo: 31.0

Claude 3 Opus: 27.3

Mistral Large: 17.7

Mistral Medium: 15.3

Gemini Pro: 14.2

Qwen 1.5 72B Chat: 10.7

Claude 3 Sonnet: 7.6

GPT-3.5 Turbo: 4.2

Mixtral 8x7B Instruct: 4.2

Llama 2 70B Chat: 3.5

Qwen 1.5 14B: 3.1

Nous Hermes 2 Yi 34B: 1.5

Notes: 0-shot. Maximum possible is 100. Partial credit is given if the puzzle is not fully solved. There is only one attempt allowed per puzzle. In contrast, humans players get 4 attempts and a hint when they are one step away from solving a group. Gemini Advanced is not yet available through the API.

What I found interesting is how this benchmark reveals a large capabilities gap between the top, large models and the rest, in contrast to existing over-optimized benchmarks.

d--b2• 2 years ago

Also these puzzles can be _really_ hard. As a French person who's lived 10+ years in English-speaking countries, I am often completely baffled. I am not sure humans would do a lot better with 0-shot.

Vecr6• 2 years ago

It's probably somewhat g loaded. I don't know how much, but someone could look at the curves (if they have access?) for the similar sub-section of an IQ test.

mike986• 2 years ago

Do you know the average or top human score / SD? Not sure if that data is available on the link or elsewhere.

Just to make sense of your result, can you show your prompt?

When you say 3 prompts and one attempt, what does that mean?

Also regarding 0-shot, did you give the LLM the instruction that is given to human by the game?

If yes, I would count that as one shot as an example of how to properly solve one example puzzle is given.

``` How to Play

Find groups of four items that share something in common.

    Select four items and tap 'Submit' to check if your guess is correct.
    Find the groups without making 4 mistakes!

Category Examples

FISH: Bass, Flounder, Salmon, Trout FIRE ___: Ant, Drill, Island, Opal

Categories will always be more specific than "5-LETTER-WORDS," "NAMES" or "VERBS."

Each puzzle has exactly one solution. Watch out for words that seem to belong to multiple categories!

Each group is assigned a color, which will be revealed as you solve ```

Thanks

QuantumGood• 2 years ago

Can you break out the partial credit score from the "solved" score?

d--b2• 2 years ago

This article does not debate the question in its title, makes ridiculous claims like “there hasn’t been any major breakthrough in AI in decades”, and does not offer any real argument.

throwaway48r7r7• 2 years ago

LLMs solve for the next word. Human intelligence solves for survival with many types of input, visual, audio etc. You can't create an AGI if you don't solve for the problems that created human GI.

keiferski4• 2 years ago

Yes I don’t think AGI (which is entirely an ill-defined concept, but put that aside) will happen until AI is embodied in the physical world.

throwaway48r7r7• 2 years ago

Absolutely. The physical world is the input that creates the feedback loop for learning.

I would propose a definition of AGI. "A model capable of effecting the physical world through speech or physical action in a manner indistinguishable from a human."

tavavex2• 2 years ago

Why not? For a hypothetical example - if we assume that simulating a human is AGI, and we have some hypothetical space-age magic tech bruteforce the problem by simulating every neuron and connection in the brain... why would being "embodied" factor into this?

mitthrowaway23• 2 years ago

Digital data is all 1s and 0s, whether it encodes words, sounds, or pictures. Why do you think transformers only work for predicting words, when they're already successfully being used for other applications as well?

throwaway48r7r7• 2 years ago

I think much like with a basic Turing machine definition compute is possible on a variety of substrates that some kind of intelligence can be created with a whole class of implementations, transformers included. Indeed the video and image input of LLMs is one of the most exiting use cases.

baq2• 2 years ago

They’re trained to optimize guessing the next word. What they solve for to get this good at predicting the next word is an open question with answers hidden in plain sight in the weight blob.

f6v2• 2 years ago

In the grand scale, human intelligence evolved over millions of years. We went from personal computers to LLMs in mere decades. I get that everyone wants Singularity now, so do I. But there’s too much over-promise and delusion.

fullstackchris• 2 years ago

What if our LLMs fail to turn into AGI?

This is nonsense statement in and of itself. Its like wondering why an orange fails to turn into a chicken.

There are SO many missing pieces an LLM just doesnt have. LLMs could certainly be a small part of some sort of AGI _system_, but they themselves can never be AGI

erezsh• 2 years ago

"We will soon be reaching the limits of hardware scaling for larger AI models"

Worth noting it's been said before for each version of GPT, only to be proven wrong.

NoGravitas• 2 years ago

There's also the question of input data, though. Current large models have been trained on all the available human-created input. Trying to add more will lead to poisoning by AI-generated data and model collapse.

Vecr6• 2 years ago

It's probable you can train the same system on the same data multiple times and still get an improvement. You could also train on universal sequence prediction data in between as well.

MacsHeadroom• 2 years ago

Model collapse is basically a myth and is a joke in the ML community. The assumptions for the model collapse paper do not hold in the real world even when training on uncrurated generated data. In fact, LLMs of equal size trained on newer web scrapes which include generated data have enhanced capabilities.

But in practice training data is curated and synthetic generated (curated) training data is even better than human data. State of the art LLMs like Phi;2 or the recent GPT-4 killer Claude 3 are trained entirely or mostly on generated data.

dondeee• 2 years ago

Come on, what’s next? Are we going to doubt that Jesus is coming, too? Not cool

DeathArrow• 2 years ago

What if AGI is not coming?

Nothing of value will be lost.

zhugeyangyang2• 2 years ago

The next generation of large models is to train several models with different specialties (small and large), and then have a front-end for task scheduling, which is then assigned to different sub models to obtain strong capabilities and professionalism while also controlling costs?

JoshCole• 2 years ago

The article claims as part of its argument that AI has not had algorithmic advances since the 80s. This is an exceedingly false premise and a common misconception among the ignorant. It would actually be fairer to say that every aspect of neural network training has had algorithmic advances than that no advances have been made.

Here is a quote from research related to this subject:

Compared to 2012, it now takes 44 times less compute to train a neural network to the level of AlexNet (by contrast, Moore’s Law would yield an 11x cost improvement over this period). Our results suggest that for AI tasks with high levels of recent investment, algorithmic progress has yielded more gains than classical hardware efficiency.

When you apply the principle of charity you can make their claim increasingly vacuous and eventually true. We're still doing optimization - we're still in the same general structure. The thing is, it becomes absurd when you do that. Its not appropriate to take such a premise seriously. It would be like taking seriously the argument that we haven't had any advancement in software engineering since bubble sort since we're still in the regime of trying to sort numbers when we sort numbers.

Its like, okay, sure, we're still sorting numbers, but it doesn't make the wider point it wants to make and its false even under the regime it wants to make the point under.

This isn't even the only issue that makes this premise wrong. For one, AI research in the 80s wasn't centered around neural networks. Hell even if you move forward to the 90s PAIP puts more emphasis on rule systems with programs like Eliza and Student than it does learning from data. So it isn't as if we're in a stagnation without advance; we moved off other techniques to the ones that worked. For another, it tries to narrow down AI research progress myopically to just particular instances of deep learning, but in reality there are a huge number of relevant advances which just don't happen to be in publicly available chat bots but which are already in the literature and force a broadening. These actually matter to LLMs too, because you can take the output of a game solver as conditioning data for an LLM. This was done in the Cicero paper. And the resulting AI has outperformed humans on conversational games as a consequence. So all those advancements are thereby advances relevant to the discussion, yet myopically removed from the context, despite being counterexamples. And in there we find even greater than 44x level algorithmic improvements. In some cases we find algorithmic improvements so great that they might as well be infinite as previous techniques could never work no matter how long they ran and now approximations can be computed practically.

somenameforme6• 2 years ago

I find a simple thought experiment answers this question. Imagine we trained an LLM using modern methods, and gave it infinite compute, on the entirety of human knowledge from 200,000 years ago. Would that AI then be able to create calculus, even if by another name obviously? I offer that as an example, because there's 0 need for knowledge of the physical world to derive calculus. All of mathematics is entirely an invention of the human mind.

I think the answer is quite obviously no. LLMs can recite their training, and recombine it in ways that correlates strongly to how a human might do so. But creating entirely new knowledge, that goes above and beyond recombinations of what is already known, remains entirely outside the domain of LLMs. An LLM trained on slow classical music is not going to create rap. And an LLM trained on rap is not going to create classical music. And those are trivial examples since it's not entirely new, but just taking a concept and using it a slightly different way than 'normal.' Math, by contrast, is literally creating something from nothing.

And this ability to create something from nothing is probably the most key indicator of intelligence. And we've yet to even step foot on the path towards creating software with this ability.

keiferski4• 2 years ago

Another way of saying this point is that intelligence requires interaction with the world, or embodiment. Hip hop can’t be derived from European classical music without the context of 70s-80s NYC.

somenameforme6• 2 years ago

We could debate that, but I think it's easier to just refer to the primary example of math. I offered that as an issue that requires absolutely no external interaction required there, whatsoever. It's entirely a mental creation. Like a child might say we somehow just 'thought it up.'

akasakahakada13• 2 years ago

Your logical loophole is that you forgot to include "process of creating thing" into the subset of human activity and this can be trained. Back in the days AlphaGo make some magical novel move in the game that shocked every Go experts. People study how AI play games to improve themselves nowadays.

LLM nowadays solve insane hard science problem, invent new algorithms that you don't see.

somenameforme6• 2 years ago

A game move, no matter how remarkable, is just a recombination within a strictly defined domain. And LLMs can do some awesome things in these very tight and well defined domains, especially when there is a way for them to determine victory or score. But stepping outside of well defined domains and creating knowledge that does not directly derive from what is already known is where intelligence shines. We somehow created mathematics out of absolutely nothing. We looked at the world around us and simply 'though it up' as a child might say.

episteme• 2 years ago

I think also interesting if you ask, what if the model consumed all information up to the point that a great discovery was made? If a model has all the information that Einstein had, could it determine that E = mc2?

freilanzer• 2 years ago

Billions to trillions of dollars will be poured into research over the next decade. More humans than ever are looking for breakthroughs. We have exponentially increased the parallel efforts. LLM architecture might be unable to deliver in its current state, but it has ignited monumental investments into research that might find other paths.

That's not really true, though. Neural network based approaches are funded, and among those mostly transformers and large language models. Real alternatives aren't funded that much, imo.

simne• 2 years ago

Article based on so many erroneous assumes, I can't believe I see it on HN!

Most important, authors don't know, that all modern AI based on Back Propagation calculations, just because they are easier to implement on cheap old hardware, but natural neurons working on Forward Propagation, which is magnitudes faster on inference.

Unfortunately, for FP we need other hardware, but it is not mean "reaching the limits of hardware scaling", it is just scaling limits for CURRENT hardware, totally other sense.

Sure, if people will play blind and avoid to see obvious things, we will have new AI winter, before somebody will reconsider FP technology.

tennisflyi• 2 years ago

It and automation have always been coming. However, it will be here one day.

cykros3• 2 years ago

It strikes me as amazing that we went from the general recognition that AGI wasn't anywhere soon to suddenly having this widespread idea that it was right around the corner.

Sort of reminds me of the late 90s super-proto-VR stuff where people thought any day now we'd be jacking into full immersion (tactile, smell and all) virtual reality.

Don't get me wrong, LLM's are useful tools. But ChatGPT aint Neuromancer. Or even Wintermute. It's Clippy after a few years of community college.

What if AGI is not coming?

Related Stories