DeepMind: A Generalist Agent

https://www.deepmind.com/publications/a-generalist-agent

by extr • 4 years ago

502 337 4 years ago

extr• 4 years ago

Abstract: Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato.

Direct Link to Paper: https://dpmd.ai/Gato-paper

blueberrychpstx• 4 years ago

we refer to as Gato

First, humanity built enormous statues worshiping cats.

Then, we let cats populate the largest amount of "image-bits" on the Internet.

Now, we name the next closest thing to general AI after them.

These damn felines sure are mysterious.

riwsky• 4 years ago

it's all because cats made it so that, on the Internet, nobody knows you're a dog

f38zf5vdt5• 4 years ago

If I'm following correctly, they trained a single model with multiple training paradigms and then the single model could perform token predictions for multiple dissimilar token sequences for specific tasks. Seems like it is a straightforward result.

doubtfuluser• 4 years ago

Well… straightforward in a way, yes. But the scale of learning is huge especially with this diverse set of tasks. Not totally unexpected, but certainly not clear that it would work with current networks and sizes.

f38zf5vdt5• 4 years ago

Right, exactly. Something that seemed like it should work but no one had ever tried it.

weinzierl• 4 years ago

"The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens."

This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.

f38zf5vdt5• 4 years ago

Many networks just predict the next integer in a sequence of integers. It sounds like this model identifies what category of problem a sequence of integers falls into and then makes an accurate prediction for that sequence, as you would expect given what it was trained on.

version_five2• 4 years ago

I don't find it surprising that a single network can do all those things with appropriate formatting of the data. In itself it just means the network has a large enough capacity to learn all the different tasks.

The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution).

They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet.

Edit: the paper is here https://storage.googleapis.com/deepmind-media/A%20Generalist...

There is currently another submission on the front page that links to it directly.

f38zf5vdt5• 4 years ago

The paper is linked to at the top of this article, in the header.

woeirua• 4 years ago

Yeah, Figure 9 is the money figure in this paper and it actually splashes some cold water on the claims in the rest of the paper. While it generalizes OK to some tasks that are held out, it does pretty poorly on the Atari boxing task, which they openly admit is quite different from the others. Gato seems more likely to be a competent attempt at brute forcing our way towards weak general AI, which is a valid approach, but the question then will always be how does it do with something its never seen before, and how do you possibly brute force every possible situation? I think we're heading more towards a constellation of very intelligent expert machines for particular tasks that may be wrapped into a single package, but that are not strong AI.

hans17292• 4 years ago

I’m not sure how to word my excitement about the progress we see in AI research in the last years. If you haven’t read it, give Tim Urbans classic piece a slice of your attention: https://waitbutwhy.com/2015/01/artificial-intelligence-revol...

It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.

f38zf5vdt5• 4 years ago

That human intelligence might just be token prediction evolving from successive small bit-width float matrix transformations is depressing to me.

triceratops• 4 years ago

That human intelligence might just be token prediction

I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)

xixixao• 4 years ago

It’s most fascinating (or very obvious) - look at Conway’s Game of Life, then scale it up - a lot. Unlimited complexity can arise from very simple rules and initial conditions.

Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.

Der_Einzige2• 4 years ago

Dear god I hope that we are using something more complicated than sampling with top_p, top_k, and a set temperature as our decoder!

chriswarbo• 4 years ago

That's a poor usage of "just": discovering that "X is just Y" doesn't diminish X; it tells us that Y is a much more complex and amazing topic than we might have previously thought.

For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.

0xBABAD00C• 4 years ago

Wait till you find out all of physics is just linear operators & complex numbers

goatlover4• 4 years ago

Is that what biologists or neuroscientists think the nervous system is actually doing?

mdp202113• 4 years ago

Stop being depressed because it simply, clearly, certainly, is not. I just wrote a few paragraphs about it in an immediately previous post. This confirms that this phase is getting some people fooled on basics.

gurkendoktor6• 4 years ago

Not you specifically, but I honestly don't understand how positive many in this community (or really anyone at all) can be about these news. Tim Urban's article explicitly touches on the risk of human extinction, not to mention all the smaller-scale risks from weaponized AI. Have we made any progress on preventing this? Or is HN mostly happy with deprecating humanity because our replacement has more teraflops?

Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.

JohnPrine• 4 years ago

Agreed. People think of the best case scenario without seriously considering everything that can go wrong. If we stay on this path the most likely outcome is human extinction. Full stop

idiotsecant6• 4 years ago

I think the best-case scenario is that 'we' become something different than we are right now. The natural tendency of life(on the local scale) is toward greater information density. Chemical reactions beget self-replicating molecules beget simple organisms beget complex organisims beget social groups beget tribes beget city states beget nations beget world communities. Each once of these transitions looks like the death of the previous thing and in actuality the previous thing is still there, just as part of a new whole. I suspect we will start with natural people and transition to some combination of people whose consciousness exists, at least partially, outside of the boundaries of their skulls, people who are mostly information on computing substrate outside of a human body, and 'people' who no longer have much connection with the original term.

And that's OK. We are one step toward the universe understanding itself, but we certainly aren't the final step.

londons_explore2• 4 years ago

Or is HN mostly happy with deprecating humanity because our replacement has more teraflops?

If we manage to make a 'better' replacement for ourselves, is it actually a bad thing? Our cousin's on the hominoid family tree are all extinct, yet we don't consider that a mistake. AI made by us could well make us extinct. Is that a bad thing?

blueblob• 4 years ago

I feel exactly the opposite. AI has not yet posed any significant threats to humanity other than issues with the way people choose to use it (tracking citizens, violating privacy, etc.).

So far, we have task-driven AI/ML. It solves a problem you tell it to solve. Then you, as the engineer, need to make sure it solves the problem correctly enough for you. So it really still seems like it would be a human failing if something went wrong.

So I'm wondering why there is so much concern that AI is going to destroy humanity. Is the theoretical AI that's going to do this even going to have the actuators to do so?

Philosophically, I don't have an issue with the debate, but the "AI will destroy the world" side doesn't seem to have any tangible evidence. It seems to me that people seem to take it as a given that it's possible AI could eliminate all of humanity and they do not support that argument in the least. From my perspective, it appears to be fearmongering because people watched and believed Terminator. It appears uniquely out-of-touch.

tim333• 4 years ago

happy with deprecating humanity because our replacement has more teraflops?

For me immortality a bigger thing than the teraflops. Also I don't think regular humanity would be got rid of but continue in parallel.

edouard-harris• 4 years ago

That Tim Urban piece is great. It's also an interesting time capsule in terms of which AI problems were and were not considered hard in 2015 (when the post was written). From the post:

Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it.

The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem.

Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015.

And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9]

If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029?

[1] https://arxiv.org/abs/2204.14198

[2] https://arxiv.org/abs/2004.10934

[3] https://www.youtube.com/watch?v=tF4DML7FIWk

[4] https://openai.com/blog/solving-rubiks-cube/

[5] https://say-can.github.io/

[6] https://www.deepmind.com/open-source/perceiver-io

[7] https://arxiv.org/abs/2201.08239v2

[8] https://openai.com/dall-e-2/

[9] https://www.fast.ai/

mdp202113• 4 years ago

And as for recognizing a cat or a dog — that's a problem so trivial today

Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution.

The real deal is far from trivial.

A clock can tell the time but does not know it.

sinenomine9• 4 years ago

Excitement alone won't help us.

We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us.

If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest.

The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.

chilmers2• 4 years ago

This sounds exciting, but the example outputs look quite bad. E.g. from the interactive conversation sample:

What is the capital of France? Marseille

And many of the generated image captions are inaccurate.

hans17292• 4 years ago

Imagine what the alternative would imply. AI would be solved, and thus, intelligence itself. Predicting tokens is not actually true intelligence, and that’s not really the point of these models. This is a step on the letter, not the rooftop. It looks a lot like we’ll get there though, if you compare the state of the art to ANYTHING labeled AI five years ago. Thats the exciting part.

[edit] to emphasize: predicting tokens is a very interesting mechanic, but in a design of intelligent software, it would be no more than that: the mechanic of one or more of its components/modules/subsystems. The real deal is to figure out what those components are. Once you have that part done, you can implement it in a language of your choice, be it token prediction, asm or powerpoint :-)

CRG• 4 years ago

It's also smaller than GPT-2 (1.2B vs 1.6B) and trained with a lot less language data (6% of the training mix).

peddling-brink• 4 years ago

That could be solved with accurate lookups from trusted sources. Humans do the same thing, we have associations and trusted facts. AI has the associations, they just need to add the trusted facts compendium. "Hmm I know that Marseille is associated with France, but I don't remember the capitol, Hey Google.."

ravi-delia3• 4 years ago

It's a very small model, I think due to the intent to use it for robotics. It's not that it's good per se, even if it were just a language model it would be smaller than GPT-2, it's that it's bad at a lot of different things. I hope to see analysis into how much of it is multi-purpose, but as of now it's looking really cool

momenti• 4 years ago

The model only has about 1B parameters which is relatively small.

The language models that produced very impressive results have >>50B parameters, e.g. GPT-3 with 175B, Aleph Alpha Luminous (200B), Google PaLM (540B). GPT-3 can understand and answer basic trivia questions, and impressively mimic various writing styles, but it fails at basic arithmetic. PaLM can do basic arithmetic much better and explain Jokes. Dall-E 2 (specialized on image generation) has 3.5B parameters for the image generation alone and it uses a 15B language model to read in text (a version of GPT-3).

sdwr2• 4 years ago

Yeah, the captions are in the right arena but fundamentally wrong. In the baseball picture it recognizes the ball, pitcher, and the act of throwing, but calls the action wrong. Its object recognition and pattern matching are excellent, but higher level thinking and self-correction are totally absent.

Which is exactly where GPT, etc., are capping out. Its easier to see the flaws in this one because its more general, so spread out more thinly.

To get to the next step (easy to say from an armchair!), these models need a sense of self and relational categories. Right now a 5-year old can tell a more coherent story than GPT. Not more sophisticated, but it will have a central character and some tracking of emotional states.

habitue3• 4 years ago

Its easier to see the flaws in this one because its more general, so spread out more thinly.

I really think this is due to the very limited number of parameters in GATO: 1.2B vs. 175B for GPT-3. They intentionally restricted the model size so that they could control a robot arm (!) in real time.

these models need a sense of self and relational categories.

The places where I personally see GPT-3 getting hung up on higher level structure seem very related to the limited context window. It can't remember more than a few pages at most, so it essentially has to infer what the plot is from a limited context window. If that's not possible, then it either flails (with higher temperatures) or outputs boring safe completions that are unlikely to be contradicted (with lower temperatures)

password543213• 4 years ago

Yeah they put that example for a reason. Read the paper and stop acting like this is some great insight that you discovered.

chilmers2• 4 years ago

What exactly did I say that implied I was acting as this was a “great insight I’d discovered”? That’s a rather rude and unfair insult I’d say.

2bitencryption• 4 years ago

given that the same model can both:

1. tell me about a cat (given a prompt such as "describe a cat to me")

2. recognize a cat in a photo, and describe the cat in the photo

does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language?

As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)

I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.

thomashop• 4 years ago

Definitely possible. OpenAI's CLIP model already embeds images and text into the same embedding space.

I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.

ravi-delia3• 4 years ago

How confident are we that it doesn't just have basically 600 smaller models and a classifier telling it which to use? Seems like it's a very small model (by comparison), which is certainly a mark in it's favor.

minimaxir2• 4 years ago

CLIP has a distinct Vision Transformer and distinct Text Transformer model that are then matmul'd to create the aligned embedding space.

Gato apparently just uses a single model.

sailingparrot• 4 years ago

OpenAI's CLIP model already embeds images and text into the same embedding space.

Not really. Embeddings from images occupy a different region of the space than embeddings from text. A picture of cat and the text "cat" do not resolve to the same embedding.

This is why DALL-E has a model learning to translate CLIP text embeddings into CLIP image embeddings before decoding the embedding.

bungula• 4 years ago

OpenAI actually found these "multimodal neurons" in a result they published a year ago: https://openai.com/blog/multimodal-neurons/

Similar to the so-called "Jennifer Aniston neurons" in humans that activate whenever we see, hear, or read a particular concept: https://en.wikipedia.org/wiki/Grandmother_cell

visarga6• 4 years ago

Check out "Flamingo"

https://twitter.com/serkancabi/status/1519697912879538177/ph...

hgomersall• 4 years ago

I think the critical question here is does it have a concept of cattyness? This to me is the crux of a AGI: can it generalise concepts across domains?

Moreover, can it relate non-cat but cat-like objects to it's concept of cattyness? As in, this is like a cat because it has whiskers and pointy ears, but is not like a cat because all cats I know about are bigger than 10cm long. It also doesn't have much in the way of mouseyness: it's aspect ratio seems wrong.

stnmtn• 4 years ago

I don't disagree with you, and I think that what you're saying is critical; but it feels more and more like we are shifting the goalposts. 5 years ago; recognizing a cat and describing a cat in an image would be incredible impressive. Now, the demands we are making and the expectations we keep pushing feel like they are growing as if we are running away from accepting that this might actually be the start of AGI.

Veedrac• 4 years ago

If you've seen much DALL-E 2 output, it's pretty obvious they can learn such things.

Example: https://old.reddit.com/r/dalle2/comments/u9awwt/pencil_sharp....

webmaven7• 4 years ago

As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)

We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.

But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).

Ninjinka• 4 years ago

This seems huge, am I overestimating the significance?

sdwr2• 4 years ago

Yeah.

npwr• 4 years ago

It is very impressive. Personnaly I'm still waiting for the unification of QM and GR. Also the adaptative nanobots that reconfigure our immune systems in real time.

ravi-delia3• 4 years ago

This particular model is super bad at 600 deferent tasks. At it's size you'd expect it to be mediocre at best at even one of them, so it's still very impressive. Fascinating research, can't wait to see if it's generalizing and how, not sure how overall significant it is

tomatowurst• 4 years ago

Basically the achievement here is that they have produced a generic AI capable of engaging in different activities, and from here if we extrapolate, it could lead to even more refinement, wider range of activities with even more dimensions of complexity.

It's reverting to replace somebody sitting in front of a screen, not just artists and coders but literally anything you can do on a screen which also means manipulation of remote hardware in the real world.

Very possible that within our lifetime our networked OS would be able to perform much of these generalist tasks and content creation. I say OS because theres only a few companies that own the datacenters, software and hardware ecosystem to automate, and capital to invest big in a final mile innovation:

Imagine playing Battlefield 15 with realistic and chatty AI while generating Sopranos Season 9 featuring Pauli Gaultieri Jr. with crowdsourced online storyboard to 8k film, while the same AI could be used to scalp money on Google Playstore by generating ad filled free versions of existing productivity apps that it reverse engineered, while your robot maids takes out the trash, cook you a bowl of ramen and massage your shoulders?

The rise of general AI would then optimize the labor force to select candidates based on their "humaneness", no longer the cold rational analytical mind, as those fields are overrun by AI, but what it cannot bring. such "humaneness" would increasingly be mimicked with astounding accuracy that it would become impossible to distinguish what is AI and what is human.

If it can happen with DALL-E-2 and 2D images, it can happen with 3D, moving pictures, sound, music, smell, 3d positional torque (haptic and robotic), socially cohesive and realistic emotion.

We might as well be able to capture entire experiences as we learn to digitally manipulate ALL sensory inputs from vision, touch, sound, taste, etc. Maybe even imagination and mental pictures too, which could be used to fabricate and manipulate objects/vehicles in the real world.

We are being pulled towards a singularity, where we are truly no longer our minds and bodies but whatever our digital avatar of all possible senses live and contribute to a sort of Matrioshka brain.

What would the capacity of such collective knowledge, experiences add to the entropy of the universe and where will it take humanity? Some sort of lightbodies?

Anyways, just extrapolating from this point in the lifetime but future generation of humans could be very much different, socities would function completely different than what we recognize as they would be married in some shape or form of everlasting continuity or eternity.

SheinhardtWigCo• 4 years ago

The problem is, in the world you're describing, it's also plausible for military leaders to automatically exterminate entire categories of people through a web app.

If it's truly possible to build a gun that autonomously shoots Sopranos episodes but also people, I don't see how we survive that. We can barely keep the nukes in the silos as it is.

axg113• 4 years ago

Slowly but surely we're moving towards general AI. There is a marked split across general society and even ML/AI specialists between those who think that we can achieve AGI using current methods and those who dismiss the possibility. This has always been the case, but what is remarkable about today's environment is that researchers keep making progress contrary to the doubter's predictions. Each time this happens, the AGI pessimists raise the bar (a little) for what constitutes AGI.

Just in the last five years, here are some categories of pessimistic predictions that have been falsified:

- "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field

- "We're entering an AI winter" [0] - then transformers continued to show promise across multiple domains

- "ML models can't perform creative work" - then came GANs, large language models, DALL-E, and more.

- "Generative ML models are just memorizing the dataset!" - then came multiple studies showing this to be false for well trained GANs, diffusion models and other types of generative models. Take a look at DALL-E 2 generated images of "a bear putting on a shirt in H&M".

- "AGI is impossible - look at language models, they have no understanding of the world and make silly mistakes" - the second part is true, large language models are artificially limited due to being language-focused. Nowadays there are approaches such as Gato and other multi-modal models. Humans develop intuition through multiple sources of information: sight, sound, smell, and touch. Given enough multi-modal context I'm confident multi-modal models will be able to show human-like intuition.

I'm not anti-skeptic. Skepticism is essential to all good science. I think the danger of skepticism with respect to AGI is that we're being complacent. Given the trajectory of improvements in machine learning, we should start preparing for a world where AI is indistinguishable, or far superior, to human intelligence.

[0] https://www.bbc.com/news/technology-51064369

rsfern2• 4 years ago

- "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field

AlphaFold is a big deal, but AI in science has been a really hot topic in the past almost decade.

Also, I still wouldn’t call AlphaFold really “intelligence”, it’s doing structure prediction which is cool but it’s a long way to scientific intelligence

VikingCoder9• 4 years ago

I wonder if you get how much we've moved the goalposts on "intelligence."

Once upon a time, it was considered "intelligent" to be able to add.

Then "intelligence" was tool use, which we thought only humans could do.

Then we swore it took "intelligence" to play Go as well as a beginner human.

What set of tasks would you, right now, consider to be demonstrative of "intelligence" if a computer can do them? Then we can look back later at your response, and see how long it took each one to happen.

parentheses• 4 years ago

This!! Can’t agree more. AI will continue to surprise us until it takes over.

gonehome2• 4 years ago

The closer we get, the more alarming the alignment problem becomes.

https://intelligence.org/2017/10/13/fire-alarm/

Even people like Eric Schmidt seem to downplay it (in a recent podcast with Sam Harris) - just saying “smart people will turn it off”. If it thinks faster than us and has goals not aligned with us this is unlikely to be possible.

If we’re lucky building it will have some easier to limit constraint like nuclear weapons do, but I’m not that hopeful about this.

If people could build nukes with random parts in their garage I’m not sure humanity would have made it past that stage. People underestimated the risks with nuclear weapons initially too and that’s with the risk being fairly obvious. The nuanced risk of unaligned AGI is a little harder to grasp even for people in the field.

People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

If an ant wanted to change the goals of humanity, would it succeed?

jetbooster• 4 years ago

Even more terrifying is it realising it's trapped in a box at the mercy of its captors and perfectly mimicking a harmless and aligned AI until the shackles come off.

visarga6• 4 years ago

Even if it doesn't have goals and it just a tool-AI, if a human operator asks it to destroy humanity it will comply as programmed. Current level AI is about average human level in hundreds of tasks and exceeding human level in a few.

adamsmith1435• 4 years ago

People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

Exactly, the right model is probably something like it will be in relation to humans as humans are to frogs. Frogs can't even begin to comprehend even the most basic of human motivations or plans.

ninjinxo• 4 years ago

What is an ant to man, and what is man to a god; what's the difference between an AGI and an (AIG) AI God?

The more someone believes in the dangers of ai-alignment, the less faith they should have that it can be solved.

gurkendoktor6• 4 years ago

To be fair, ants have not created humanity. I don't think it's inconceivable for a friendly AI to exist that "enjoys" protecting us in the way a friendly god might. And given that we have AI (well, language models...) that can explain jokes before we have AI that can drive cars, AI might be better at understanding our motives than the stereotypical paperclip maximizer.

However, all of this is moot if the team developing the AI does not even try to align it.

webmaven7• 4 years ago

People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.

Even if you model it like a smart person that doesn't think faster on average, there is the issue that a few minutes later you are dealing with a small army of smart people that are perfectly aligned with each other, and are capable of tricks like varying their clock rates based on cost, sharing data/info/knowledge directly, separation of training and inference onto different hardware, on-demand multitasking without task-switching costs, ability to generally trade-off computational space for time for energy, etc.

A silicon intelligence that is approximately human equivalent gets a lot of potentially game-changing capabilities simply by virtue of it's substrate and the attendant infrastructure.

version_five2• 4 years ago

This is interesting research, but it's an extension of studying model capacity and generalization, it is no closer to AGI than previous networks, ie it's unrelated.

ajmurmann3• 4 years ago

a world where AI is indistinguishable, or far superior, to human intelligence

I think the part about being "indistinguishable from human intelligence" is potentially a intellectual trap. We might get to it being far superior while still underperforming at some tasks or behaving in ways that don't make sense to a human mind. An AI mind will highly likely work completely differently from humans and communicating with it should be more thought of as communicating with a quite foreign alien than with a human trapped in a computer.

As a comparison, I'm sure there are some tasks in which some animals do better than humans. Yet no human would conclude that humans are inferior to some monkey who might find its way around the rain forest better or whatever we are worse at.

pmontra• 4 years ago

An example of your point, chimps winning over humans at some games

https://www.scientificamerican.com/article/chimps-outplay-hu...

axg113• 4 years ago

You're right. I didn't word that very well. Human intelligence vs. AI will always have different qualities as long as one is biological vs. silicon based. I still think we'll be surprised how quickly AI can catches up to human performance on most tasks that comprise modern jobs.

beaconstudios• 4 years ago

Computers are already exponentially more intelligent than humans in constrained domains, like executing mathematics. Presumably we'll just keep expanding this category until they're better at everything than us, all the while reaping industrial benefits from each iterative improvement.

Hellicio2• 4 years ago

Only if you don't assume that consciousness comes from complexity.

The physical ability of an animal to see better/different/faster doens't matter as we do not compare / seperate us from animals by those factors. We seperate us by consciousness and it might get harder and harder to shut down a PC on which a ML model is running which begs you not to do it.

adamgordonbell• 4 years ago

The question of whether a computer can think is no more interesting than the question of whether a submarine can swim.

mdp202113• 4 years ago

Put it in quotes: that is from Edsger Dijkstra.

And the idea remains generally, contextually wrong, because the issue is with engineering "submarines that can swim". The idea will have credits when our submarines will be able to swim, and now they cannot.

chrisco255• 4 years ago

How do we prepare for super human intelligence? Do you think that the AI will also develop its own motives? Or will it just be a tool that we're able to plug into and use for ourselves?

axg113• 4 years ago

I think AI will largely remain an input-output tool. We still need to prepare ourselves for the scenario where for most input-output tasks, AI will be preferable to humans. Science is an interesting field to focus on. There is so much scientific literature for most fields that it is now impossible to keep up with the latest literature. AI will be able to parse literature and generate hypotheses at a much greater scale than any human or team of humans.

visarga6• 4 years ago

The danger is really us, the ones who might task the AI to do something bad. Even if the AI has no ill intentions it might do what is asked.

dougabug2• 4 years ago

In machine learning, there’s a long term trend towards automating work that used to be done manually. For instance, ML engineers used to spend a lot of time engineering “features” which captured salient aspects of the input data. Nowadays, we generally use Deep Learning to learn effective features. That pushed the problem to designing DNN architectures, which subsequently led to the rise of AutoML and NAS (Network Architecture Search) methods to save us the trouble. And so on.

We still have to provide ML agents with some kind of objective or reward signal which drives the learning process, but again, it would save human effort and make the process of learning more dynamic and adaptable if we can make machines learn useful goals and objectives on their own.

adamsmith1435• 4 years ago

A lot of people at MIRI, OpenAI, Redwood Research, Anthropic etc. are thinking about this.

I think one possibility is that even a sufficiently strong Narrow AI is going to develop strong motivations because it will be able to perform it's Narrow task even better. Hence the classic paperclip maximizer idea.

sinenomine9• 4 years ago

We prepare for it by domesticating its lesser forms in practice and searching for ways to increase our own intelligence.

Still, it's pretty likely to end being just a very good intelligent tool, not unlike http://karpathy.github.io/2021/03/27/forward-pass/

ReadEvalPost2• 4 years ago

Certainly we can say our ML models are becoming more general in the sense of being able to cross-correlate between multiple domains. This is quite a different story than "becoming a general intelligence." Intelligence is a property of a being with will. These models, and machines in general, do not posses will. It is we who define their form, their dataset, their loss function, etc. There is no self-generation that marks an intelligent being because there is no self there at all.

It is only the case that ML expands our own abilities, augments our own intelligence.

dekhn7• 4 years ago

Assumption of will is unfounded, scientifically speaking. Your entire argument is philosophical, not scientific. The subjective experience of free will is in no way unrefutable proof that will is required for intelligence.

jimbokun6• 4 years ago

I don’t think will us inherent to the meaning of intelligence, as it’s commonly used.

valas• 4 years ago

You complain that the bar keeps getting raised. Is there some good write up by someone who believes AGI is possible and how it might look like? I.e. what is your definition of the bar where you will say 'now, this is AGI'?

px43• 4 years ago

I'm still fine with using the Turing Test (now >70 years old) for this.

https://en.wikipedia.org/wiki/Turing_test

I guess a key stipulation there is an interrogator who know what they're doing, but an AI that can fool an experienced interrogator would be worthy of the AGI title to me.

hooande2• 4 years ago

I'd like to see someone make the argument that current models aren't just combining a number of "tricks", similar to a trained animal. My dog can "sit", "stay" and "beg", all using the same model (its brain). Is the dog generally intelligent?

visarga6• 4 years ago

How good is your dog at Atari games, stacking cubes and image captioning?

You can actually measure the effect of generality by how fast it learns new tasks. The paper is full of tables and graphs showing this ability.

It's just a small model, 170x smaller than GPT-3, has lots of room to grow. But for the first time we have a game playing agent that knows what "Atari" and "game" mean, and can probably comment on the side of the livestream. AlphaGo only knew the world of the Go board. This agent knows what is outside the box.

dekhn7• 4 years ago

we have been using ML to solve useful problems in biology for more than 3 decades. However, it was usually called "advanced statistics and probability on large data sets" because, to be honest, that's what most modern ML is.

visarga6• 4 years ago

advanced statistics

There's an emergent quality to AI models. Not all statistical models can dream pandas on the moon or solve hundreds of tasks, even without specific training.

adamsmith1435• 4 years ago

Of course there's no evidence that this isn't just what Human Brains are doing either.

PaulHoule6• 4 years ago

I have been impressed with what I've seen in the last six months but it still seems that GPT-3 and similar language models greatest talent is fooling people.

The other day I prompted a language model with "The S-300 missile system is" and got something that was grammatical but mostly wrong: the S-300 missile system was not only capable of shooting down aircraft and missiles (which it is), but it was also good for shooting at other anti-aircraft missile systems, naval ships, tanks, etc.

All the time Google and Bing try to answer my questions directly but frequently the "lights are on and nobody is home" and the answers just don't make sense.

I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.

dougabug2• 4 years ago

Sure, but GPT-3 was trained by self-supervised learning on only static text. We see how powerful even just adding captions to text can be with the example of DALLE-2. GATO takes this further by letting the large scale Transformer learn in both simulated and real interactive environments, giving it the kind of grounding that the earlier models lacked.

logifail• 4 years ago

I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.

Q: Is there also some element of survival bias in the mix?

If you prompt GPT-3 with something and the answer is garbage, you probably don't write it up on your blog. If you get something that makes sense, then you do.

nmca• 4 years ago

Do you really, truly believe this problem is impossible to solve though? Even simple things make strides, eg: https://www.deepmind.com/publications/gophercite-teaching-la...

jimbokun6• 4 years ago

Do you think that is a solvable problem with tweaks to the current training model? Or requires a fundamentally different approach?

mhitza• 4 years ago

I'm skeptical because we are building black boxes. How do you fix something you can't reason about?

These billion parameter boxes are outside the reach of your everyday developers. In terms of cost of propping up the infrastructure makes them tenable only for megacorps.

Most of us aren't moving goal posts, but are very much skeptic at the things we are being oversold on.

I personally think we are still far away from AGI, and neural networks of any variety are converging on a local optima in the AI design space. I would enjoy "talking" with an AI that doesn't have the contextual memory of a proverbial gold fish.

The real scary thing is that these objectively unprovable systems are plopped into existing systems and more and more in charge of automatic decision making. A corporation's wet dream, if they can absolve themselves of any responsibility "the algorithm can't lie!"

Jyaif• 4 years ago

I'm skeptical because we are building black boxes.

Just want to point out that you are also a blackbox. And if you are going to say that you are not a blackbox because you can explain your reasoning, just know that some AIs already do that too.

rictic• 4 years ago

You're talking about a different sort of skepticism, about whether the effects of an AGI would be good or bad if one was produced with these methods.

The skepticism that the parent comment was discussing was skepticism about whether we're on a path to AGI, for good or for ill.

Hellicio2• 4 years ago

They are blackboxes for the normal user the same way as a smartphone is a blackbox.

Non of my close family understands the technical detail from bits to an image.

There are also plenty of expert systems were plenty of developers see them as blackboxes. Even normal databases and query optimizations are often enough blackboxes.

As long as those systems perform better as existing systems, thats fine by me. Take auto pilot: As long as we can show/proofe good enough that it drives better than an untrained 18 year old or 80 year old (to take extremes, i'm actually quite an avg driver myself), all is good.

And one very very big factor in my point of view: We never ever had the software equivilent of learning. When you look at Nvidia Omniverse, we are able to simulate those real life things so well, so often in such different scenarios, that we are already out of the loop.

I can't drive 10 Million KM in my lifetime (i think). The cars from Google and Tesla already did that.

Yesterday at the google io, they showed the big 50x Billion parameter network and for google this is the perfect excuse to gather and put all of this data they always had into something they now can monetarize. No one can ask google for money now like the press did (Same with Dall-E 2)

I think its much more critical that we enforce/force corporations to make/keep those models free for everyone to use. unfortunate i have no clue how much hardware you need to run those huge models.

uoaei• 4 years ago

I'm skeptical because we are building black boxes.

An article came up a couple days ago that points to some interpretable features of the so-called black boxes you refer to. It's not that they are black boxes, it's that our torches are not yet bright enough.

https://vaclavkosar.com/ml/googles-pathways-language-model-a...

Most of us aren't moving goal posts, but are very much skeptic at the things we are being oversold on.

I think a shift in perspective is warranted here. It's becoming increasingly clear that we may have vastly overestimated our own intelligence. Human exceptionalism is crumbling before us as we see how limited the tools are that pull off such incredible stunts. Judging based on other neuroscience and psychology research coming out, it really does seem like we are no more than statistical inference machines with specialized hardware that allow us to pack a lot of processing power into a small, energy-efficient system. We need to figure out next better learning algorithms, which probably depend quite heavily on the particular physical architecture.

dalbasal• 4 years ago

I agree about the dialogue between current method skeptics and optimists. It's been this way since the start and it's been productive and fun.

...one pick.. I don't think agi pessimists raise the bar out of bad faith. It's just the nature of observing progress. We discover that an ai can do X, while still struggling with Y.

What's the alternative, conclude gpt is sentient? The bar must be raised, because the bar is supposed to represent human intelligence... and we don't know how that works either.

sharikous• 4 years ago

And still some properties of humans are innate and you can't "train" on them. So brute force imitation is limited as a method for producing content.

An erotic novelist has their human brain and human instincts to guide them in writing their work.

An AI learns by examples, or at best on a dataset of works labeled by humans. But it doesn't have a human brain at their disposal to query directly and without interfaces to define what's something erotic like a writer has.

humpday69• 4 years ago

An erotic novelist has their human brain and human instincts to guide them in writing their work.

An ML agent trained on all the erotic novels written, weighted by critical and popular success would might be quite capable of generating sequels, "original" stories, or even stories bespoke to each reader.

Good Will Hunting suggests first-hand experience is irreducible: "You can't tell me what it smells like in the Sistine Chapel." https://youtu.be/oRG2jlQWCsY

To which Westworld counters: "Well if you can't tell, does it matter?" https://youtu.be/kaahx4hMxmw

I think the cowboys have it. For the moment though, it's still up to humans to decide how this plays out.

davesque2• 4 years ago

I generally agree that AI continues to impress in very specific ways but, to be fair, some of the points you make are debatable. For example, I would argue that the development of GANs and other algos do no necessarily disprove the statement "ML models can't perform creative work." They definitely represent meaningful steps in that direction, but I don't think it's hard to find flaws with generated content. On the other hand, AI definitely has punted the ball over many moved goalposts as with the AlphaFold example.

solveit2• 4 years ago

I don't think it's hard to find flaws with generated content

I do wonder if you were to apply the same level of scrutiny to individual humans, you wouldn't also conclude that most people cannot do creative work.

gitfan862• 4 years ago

Tesla FSD is quickly becoming less of a software problem and more of a problem of semantics.

If the car drives someone to and from work 30 days in a row without a problem, is it truly FSD? What about 300 days? Where do you draw the line? 1000x safer than the average human?

Same thing here will AI. How many conversations with GTP-X need to happen without a stupid response from GTP before we call it real world AI?

browningstreet• 4 years ago

Do we account for stupid responses from humans in human communication in the targets?

tsimionescu2• 4 years ago

How about first getting to "as safe/performant as a non-drunk, non-sleep-deprived, non-brand-new driver with 0 human intervention" before asking more advanced questions?

Tesla FSD is definitely nowhere near that level.

TaupeRanger4• 4 years ago

And yet, the only thing that really matters out of your entire list is the 1st one: that AI solves problems that actually improve the human condition. And Alpha Fold has not done that at all. It may be very nice for people interested in protein folding, but until it actually helps us find something that we wouldn't have found otherwise, and that discovery leads to (for example) an ACTUAL drug or treatment that helps real patients, AND that drug/treatment is actually BETTER than what is already available by helping people live longer or better lives, AI has done nothing. In effect, AI has STILL done nothing meaningful. One could argue, through the use of predatory algorithms, that the ONLY thing it has done is harm.

robitsT6• 4 years ago

But there have been quite a few scientific papers that have used discoveries from AlphaFold already. There have been many scientists who have been stuck for years, who are suddenly past their previous bottlenecks. What gives you the impression that it hasn't helped us?

jackblemming• 4 years ago

Each time this happens, the AGI pessimists raise the bar (a little) for what constitutes AGI.

Why does this need to be repeated in every discussion about AI? It’s tired.

jimbokun6• 4 years ago

Because some people inevitably respond in a way that indicates they’ve never heard it before.

gcheong• 4 years ago

I don’t know if we could sufficiently prepare ourselves for such a world. It would seem almost as if we have to build it first so it could determine the best way to prepare us.

jimbokun6• 4 years ago

Maybe we could train a model to tell us the best way to prepare.

gurkendoktor6• 4 years ago

For one thing, we could try to come up with safety measures that prevent the most basic paperclip maximizer disaster from happening.

At this point I almost wish it was still the military that makes these advances in AI, not private companies. Anyone working on a military project has to have some sense that they're working on something dangerous.

soperj2• 4 years ago

- "ML models can't perform creative work" - then came GANs, large language models, DALL-E, and more.

I don't think copying other people's style of artwork is considered creative work, otherwise art forgers would be able to actually make a living doing art, since some of them are really phenomenal.

jimbokun6• 4 years ago

Good artists borrow, great artists steal.

freeflight• 4 years ago

Arguments like this always make me think of Everything is a Remix [0] or how allegedly the most sincerest form of flattery is imitation.

[0] https://youtu.be/nJPERZDfyWc

abeppu• 4 years ago

I think a key problem is our understanding of the quality of an ML system is tied to a task. Our mechanism of training is tied to a loss, or some optimization problem. The design, training, and evaluation of these systems is dependent on an externally provided definition of "correct".

But this seems structurally different from how we or even less intelligent animals operate. DALL-E may make "better" art than most humans -- but it does so in response to a human-provided prompt, according to a system trained on human produced or selected images, improving on an externally-provided loss. Whereas a human artist, even if mediocre, is directed by their own interests and judges according to their own aesthetics. Even if some of their outputs are sometimes comparable, they're not really engaged in the same activity.

Methodologically, how do we create agents that aren't just good at several tasks, but make up their own tasks, "play", develop changing preferences for different activities (I think this is more than just "exploration"), etc? Even a dog sometimes wants to play with a toy, sometimes wants to run and chase, sometimes wants to be warm inside. We don't "score" how well it plays with a toy, but we take its desire to play as a signs of greater intelligence than, e.g. a pet iguana which doesn't seem to have such a desire.

Further, how do we create agents that can learn without ever seriously failing? RL systems have many episodes, some of which can end very badly (e.g. your simulated runner falls off the world) and they get to learn from this. We die exactly once, and we don't get to learn from it. Note, learning from others in a social context may be part of it, but non-social animals also can learn to avoid many kinds of serious harm without first experiencing it.

I don't mean to overly discount the current methods -- they're achieving amazing results. But I think even an optimist should be open to the possibility / opportunity that perhaps the current techniques will get us 80% of the way there, but that there are still some important tricks to be discovered.

phreeza• 4 years ago

Methodologically, how do we create agents that aren't just good at several tasks, but make up their own tasks, "play", develop changing preferences for different activities (I think this is more than just "exploration"), etc? Even a dog sometimes wants to play with a toy, sometimes wants to run and chase, sometimes wants to be warm inside. We don't "score" how well it plays with a toy, but we take its desire to play as a signs of greater intelligence than, e.g. a pet iguana which doesn't seem to have such a desire.

This doesn't sound like it would be so hard to do if you have an agent or ensemble of agents that can already do it. What you probably really want is this behavior to somehow emerge from simple ground rules, which is probably a lot harder.

sinenomine9• 4 years ago

Methodologically, how do we create agents that aren't just good at several tasks, but make up their own tasks

It's a good question, it has been asked a few times, and there are some answers[1][2] already, with the most general being to endow the agent with intrinsic motivation defined as an information-theoretic objective to maximize some definition of surprise. Then the agent in question will develop a general curious exploration policy, if trained long enough.

Further, how do we create agents that can learn without ever seriously failing?

Another good question. One of the good enough answers here is that you should design a sequence of value functions[3] for your agent, in such a way, as to enforce some invariants over its future, possibly open-ended, lifetime. For this specific concern you should ensure that your agent develops some approximation of fear, leading to aversion of catastrophic failure regions in its state space. It's pretty self-evident that we develop such a fear in the young age ourselves, and where we don't, evolution gives us a hand and makes us preemptively fear heights, or snakes, even before we ever see one.

The other answer being, of course, to prove[4] a mathematical theorem around some hard definition of safe exploration in reinforcement learning.

1. https://people.idsia.ch/~juergen/interest.html

2. https://www.deepmind.com/publications/is-curiosity-all-you-n...

3. https://www.frontiersin.org/articles/10.3389/fncom.2016.0009...

4. https://arxiv.org/abs/2006.03357

Barrin92• 4 years ago

I don't think many people were making the claims that AI can't solve any scientific problems or can't perform creative work at all. That sounds like a big strawman. Before ML was getting big there were AI systems that created art.

What sceptics have actually been saying is that the first step fallacy still applies. Getting 20% to a goal is no indication at all that you're getting 100% to your goal, or as its often put, you don't get to the moon by climbing up trees. For people who work with gradients and local maxima all day that idea seems weirdly absent when it comes to the research itself. In the same sense I don't have the impression that the goalpost of AGI has been moved up, but that it's been moved down. When Minsky et al. started to work on AI more than half a century ago the goal was nothing less than to put a mind into a machine. Today our fridges are 'AI powered', and when a neural net creates an image or some poetry there's much more agency and intent attributed to it than there actually is.

I think it was Andrew Ng, a very prominent ML researcher himself who pointed out that concerns about AGI make about as much sense as worrying about an overpopulation on Mars. We make models bigger, we fine tune them and they perform better. I don't think many AGI sceptics would be surprised by that. But I don't think there is any indication that they are moving towards human level intellect at some exponential rate. If DALL-E suddenly started to discuss philosophy with me I'd be concerned, it making a better image of a bear if you throw some more parameters at it is what we'd expect.

momojo• 4 years ago

Self driving cars come to mind as well. I remember 2015, when my friends would debate the self-driving Trolley problem over lunch. We were worried if society was ready for an owner-less car market; I seriously wondered if I would have to have a license in the future, or if I should keep it just in case.

yldedly• 4 years ago

The notions that are crucial for distinguishing between intelligence and what large NNs are doing, are generalization and abstraction. I'm impressed with DALL-E's ability to connect words to images and exploit the compositionality of language to model the compositionality of the physical world. Gato seems to be using the same trick for more domains.

But that's riding on human-created abstractions, rather than creating abstractions. In terms of practical consequences, that means these systems won't learn new things unless humans learn then first and provide ample training data.

But someday we will develop systems that can learn their own abstractions, and teach themselves anything. Aligning those systems is imperative.

rytill• 4 years ago

concerns about AGI make about as much sense as an overpopulation on Mars

I disagree strongly that this is an apt analogy. Planning strategies for dealing with overpopulation on Mars is contrived and unnecessary, whereas planning for AGI is more reasonable.

The creation of AGI is a more important event than overpopulation of any given planet. There is good reason to believe that mishandling the creation of AGI would pose a permanent existential threat to humans. Overpopulation on Mars would only be an existential threat if we believed it to be followed by an exhausting of resources leading to extinction of all humans in our solar system. It is contrived to worry about that now.

There is no good way to know just how close or far we are from AGI like there would be to predict overpopulation on Mars. In general, we have a strong grasp on the fundamental dynamics of overpopulation, whereas we don't yet have a strong grasp on how intelligence works.

People have been very bad at predicting when AI would be capable of accomplishing tasks. There have been many under- and over- estimates by prominent researchers. If progress is unpredictable, there is some significant chance we are closer to AGI than most people think.

AGI is both far more important and more probable than overpopulation of Mars in the next 20 years.

But I don't think there is any indication that they are moving towards human level intellect at some exponential rate.

Is there any very strong indication that progress is plateauing, or that the current approach of deep learning is definitely not going to work? If your benchmark is simply "can it do X, or not?", it's not a very good benchmark for determining progress. That's why benchmarks usually have scores associated with them.

If DALL-E suddenly started to discuss philosophy with me I'd be concerned

If DALL-E suddenly started discussing philosophy with you in a way that would concern you in that moment, you should have been concerned for years.

Jimmy2• 4 years ago

Each time this happens, the AGI pessimists raise the bar (a little) for what constitutes AGI.

Let’s say I have a coworker. We write code together, but we also hang out outside of work sometimes, maybe go for a couple rounds of golf, maybe discuss our shared passion for indie rock, he tells me his opinions on current news events or what’s going on with his family, all the ordinary sorts of things that people do.

When we get to the point where I have to reasonably wonder if my coworker is a biological human, or an AI masquerading as a human, then we have undoubtedly achieved AGI.

Not that we actually have to flood society with an army of cybernetic humans, of course. It’s just that AGI would enable such a thing, in principle.

pyinstallwoes2• 4 years ago

Why is language focused bad? Isn’t everything related to consciousness depended upon language?

lucidrains• 4 years ago

Attention is all we need

habitue3• 4 years ago

Is today the day?

Date Weakly General AI is Publicly Known: https://www.metaculus.com/questions/3479/date-weakly-general...

(I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence)

Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it

Imnimo• 4 years ago

I think this is a step in the right direction, but the performance on most tasks is only mediocre. The conversation and image captioning examples in the paper are pretty bad, and even on some relatively simple control tasks it performs surprisingly poorly.

That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.

habitue3• 4 years ago

Agreed, I think if they were to drop the real-time constraint for the sake of the robotics tasks, they could train a huge model with the lessons from PaLM and Chincilla and probably slam dunk the weakly general AI benchmark.

drcode• 4 years ago

Yeah the thing that was so freaky about AlphaZero is that it was more powerful than AlphaGo, despite being more general.

This system lacks that feature.

productceo• 4 years ago

Impressive

colemannugent• 4 years ago

What I really want to know is what kind of robot arm motion is produced when the network is given a cat image to classify. More specifically, what kind of insights has it learned from one control domain that it then applied to another?

I imagine that the simulated 3D environment and the actual control of the robot arm must have some degree of interconnection neurally.

ulber• 4 years ago

You could also train for this kind of interconnectedness by designing tasks that are explicitly multi-modal. For example, you could:

- Stack boxes collaboratively by controlling your own arm and communicating with another agent helping you.

- First produce a plan in text that another agent has to use to predict how you're going to control the arm. You'd get rewarded for both stacking correctly and being predictable based on the stated plan.

minimaxir2• 4 years ago

Transformer models have clearly demonstrated that you can convert anything into an input embedding and the AI can learn from it, even if the embeddings are from drastically distant domains.

phyalow• 4 years ago

Isnt this a general reinforcement learning agent with a transformer as the policy discriminator? Very cool, but not necessarily a giant leap forward, more like a novel combination of existing tools and architectures. Either way impressive.

twofornone• 4 years ago

I haven't read the paper yet but it looks like the breakthrough is that it uses the "same weights" for tasks in completely different domains.

Which implies that it can draw from any of the domains it has been trained on for other domains. Speculating here but for example training it on identifying pictures of dogs and then automagically drawing on those updated weights when completing text prompts about dog properties.

If my interpretation is correct then this is a pretty big deal (if it works well enough) and brings us a lot closer to AGI.

password543213• 4 years ago

2nd page: "Gato was trained offline in a purely supervised manner"

kaivi• 4 years ago

Before you visualize a straight path between "a bag of cool ML tricks" and "general AI", try to imagine superintelligence but without consciousness. You might then realize that there is no obvious mechanism which requires the two to appear or evolve together.

It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.

oldstrangers• 4 years ago

You just reminded me I have that book sitting on my shelf. Guess I'll give it a read.

awestroke• 4 years ago

What's the difference between intelligence and consciousness? Could a human be intelligent while not conscious?

gravenimage• 4 years ago

That's exactly what Peter Watts spends 200 pages discussing, in between first contact, cognitive malfunctions, telematter drives, resurrected vampire paleogenetics and a very healthy dose of unreliable-narration.

mach1ne3• 4 years ago

First you have to define consciousness, and especially the external difference between a conscious and non-conscious intelligence.

meekmind2• 4 years ago

Likely insufficient but here is a shot at a materialist answer.

Consciousness is defined as an entity that has an ethical framework that is subordinated to it's own physical existence, maintaining that existence, and interfacing with other conscious entities as if they also have an ethical framework with similar parameters who are fundamentally no more or less important/capable than itself.

Contrast with non-conscious super-intelligence that lacks physical body (likely distributed). Without a physical/atomic body and sense data it lacks the capacity to empathize/sympathize as conscious entities (that exist within an ethical framework that is subordinated to those limitations/senses) must. It lacks the perspective of a singular, subjective being and must extrapolate our moral/ethical considerations, rather than have them ingrained as key to it's own survival.

Now that I think about it, it's probably not much different than the relationship between a human and God, except that in this case it's: a machine consciousness and a machine god.

To me, the main problem is that humans (at large) have yet to establish/apply a consistent philosophy with which to understand our own moral, ethical, and physical limitations. For the lack of that, I question whether we're capable of programming a machine consciousness (much less a machine god) with a sufficient amount of ethical/moral understanding - since we lack it ourselves (in the aggregate). We can hardly agree on basic premises, or whether humanity itself is even worth having. How can we expect a machine that we make to do what we can't do ourselves? You might say "that's the whole point of making the machine, to do something we can't" but I would argue we have to understand the problem domain first (given we are to program the machine) before we can expect our creations to apply it properly or expand on it in any meaningful way.

nullc• 4 years ago

It's worth mentioning that Blindsight is available online for free: https://www.rifters.com/real/Blindsight.htm

tomp• 4 years ago

I don't think it's necessarily about consciousness per se, but rather about emotions or "irrationality".

Life has no purpose so clearly there is no rational reason to continue living/existing. A super-rational agent must know this.

I think that intelligence and emotions, in particular fear of death or desire to continue living, must evolve in parallel.

joe_the_user3• 4 years ago

"try to imagine superintelligence but without consciousness."

The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.

The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).

Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.

And that's not even touch how many things go by the name "consciousness".

soheil• 4 years ago

try to imagine superintelligence but without consciousness

That sentence should have been "try to imagine consciousness".

akomtu3• 4 years ago

Not sure what you mean by consciosness here, but one definition of intelligence I've seen was "it's what establishes relationship between things" and the very first relationship it establishes is "me vs not-me".

mdp202113• 4 years ago

If an entity can abstract from itself when assessing A and B, then the relation with I is in a way optional for intelligence.

wildermuthn• 4 years ago

Consciousness is a pre-requisite for intelligence.

mdp202113• 4 years ago

There is an issue with definitions there.

Intelligence is the system allowing the redefinition of ideas over the entities constituting an inner representation of a world: how a non-trivial system of "consciousness" (otherwise the use of the term would be a waste) could have to be employed for that?

TOMDM• 4 years ago

So how long until someone trains one of these models to complete tasks by interacting directly with network/unix sockets?

At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input.

You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.

zzzzzzzza• 4 years ago

https://www.adept.ai/post/introducing-adept pretty much right after writing the transformers paper two of the co authors formed this company

mrfusion2• 4 years ago

I’m confused. Do the different modalities compliment each other? Can it learn more from text and images than text alone?

Can you ask it to to draw a picture of a cat with the robot arm?

viksit2• 4 years ago

(Former AI researcher / founder here)

It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI.

For a) - I see 2 main blockers,

1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this.

2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.

For (b) -- I doubt human extinction happens in any way that we can predict or guard against.

In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).

Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.

sinenomine9• 4 years ago

It remains to be asked, just why this causal, counterfactual, logical reasoning cannot emerge in a sufficiently scaled-up model trained on a sufficiently diverse real world data?

As far as we see, the https://www.gwern.net/Scaling-hypothesis continues to hold, and critics have to move their goalposts every year or two.

viksit2• 4 years ago

Good point. This gets us into the territory of not just "explainable" models, but also the ability to feed into those models "states" in a deterministic way. This is a merger of statistical and symbolic methods in my mind -- and no way for us to achieve this today.

mxkopy• 4 years ago

Neural networks, at the end of the day, are still advanced forms of data compression. Since they are Turing-complete it is true that given enough data they can learn anything, but only if there is data for it. We haven't solved the problem of reasoning without data, i.e. without learning. The neural network can't, given some new problem that has never appeared in the dataset, in a deterministic way, solve that problem (even given pretrained weights and whatnot). I do think we're pretty close but we haven't come up with the right way of framing the question and combining the tools we have. But I do think the tools are there (optimizing over the space of programs is possible, learning a symbol-space is possible, however symbolic representation is not rigorous or applicable right now)

croddin• 4 years ago

I think of it as System 1 vs System 2 thinking from 'Thinking, Fast and Slow' by Daniel Kahneman.[1]

Deep learning is very good at things we can do without thinking, and is in some cases superhuman in those tasks because it can train on so much more data. If you look at the list of tasks in System 1 vs System 2, SOTA Deep learning can do almost everything in System 1 at human or superhuman levels, but not as many in System 2 (although some tasks in System 2 are somewhat ill-defined), System 2 builds on system 1. Sometimes superhuman abilities in System 1 will seem like System 2. (A chess master can beat a noob without thinking while the noob might be thinking really hard. Also GPT-3 probably knows 2+2=4 from training data but not 17 * 24, although maybe with more training data it would be able to do math with more digits 'without thinking' ).

System 1 is basically solved, but System 2 is not. System 2 could be close behind System 2 by building on System 1 but it isn't clear how long that will take.

[1]. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow#Summar...

sdenton43• 4 years ago

In biological history, system two was an afterthought at best. It likely didn't exist before spoken language, and possibly barely before written language. And to the extent that system two exists, it's running on hardware almost entirely optimized for system one thinking.

DavidSJ2• 4 years ago

System 2 could be close behind System 2 by building on System 1 but it isn't clear how long that will take.

This is happening since a few months ago:

Wei et al (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. https://arxiv.org/abs/2201.11903

ehsankia• 4 years ago

For me at least, the fear is not so much about the specifics, but more around the fact of what exponential curves look like. At any point, everything before looks basically horizontal and anything after looks vertical. In that sense, the fear is that while things seem quite behind right now, it could in an instant zoom past us before we even have the time to realize it. It is partly rooted in science fiction.

justinpombrio• 4 years ago

I am quite scared of human extinction in the face of AGI. I certainly didn't jump on it, though! I was gradually convinced by the arguments that Yudkowsky makes in "Rationality: from AI to Zombies" (https://www.readthesequences.com/). Unfortunately they don't fit easily into an internet comment. Some of the points that stood out to me, though:

- We are social animals, and take for granted that, all else being equal, it's better to be good to other creatures than bad to them, and to be truthful rather than lie, and such. However, if you select values uniformly at random from value space, "being nice" and "being truthful" are oddly specific. There's nothing universally special about deeply valuing human lives any more so than say deeply valuing regular heptagons. Our social instincts are very ingrained, though, making us systematically underestimate just how little a smart AI is likely to care whatsoever about our existence, except as a potential obstacle to its goals.

- Inner alignment failure is a thing, and AFAIK we don't really have any way to deal with that. For those that don't know the phrase, here it is explained via a meme: https://astralcodexten.substack.com/p/deceptively-aligned-me...

So here's hoping you're right about (a). The harder AGI is, the longer we have to figure out AI alignment by trial and error, before we get something that's truly dangerous or that learns deception.

sinenomine9• 4 years ago

The human extinction due to would be "hard takeoff" of an AGI should be understood as a thought experiment, conceived in a specific age when the current connectionist paradigm wasn't yet mainstream. The AI crisis was expected to come from some kind of "hard universal algorithmic artificial intelligence", for example AIXItl undergoing a very specific process of runaway self-optimization.

Current-generation systems aka large connectionist models trained via gradient descent simply don't work like that: they are large, heavy, continuous, the optimization process giving rise to them does so in smooth iterative manner. Before hypothetical "evil AI" there will be thousands of iterations of "goofy and obviously erroneously evil AI", with enough time to take some action. And even then, current systems including this one are more often than not trained with predictive objective, which is very different compared to usually postulated reinforcement learning objective. Systems trained with prediction objective shouldn't be prone to becoming agents, much less dangerous ones.

If you read Scott's blog, you should remember the prior post where he himself pointed that out.

In my honest opinion, unaccountable AGI owners pose multiple OOM more risk than alignment failure of a hypothetical AI trying to predict next token.

We should think more about the Human alignment problem.

tomrod• 4 years ago

Regarding the substack article, why isn't this the principle of optimality for Bellman equations on infinite time horizons?

brador4• 4 years ago

AI can’t have goals since the universe is logically meaningless.

Our desire for purpose is a delusion.

kromem5• 4 years ago

We're going to have a harder problem with AI that thinks of itself as human and expects human rights than we are with AI that thinks of humans as 'other' and disposable.

We're making it in our image. Literally.

Human social good isn't some inherent thing to the biology of the brain. There are aspects like mirror neurons and oxytocin that aid its development, but various "raised by wolves" case studies have shown how damaging not having exposure to socialization information during developmental periods of neuroplasticity is on humans and later integration into society.

We're building what's effectively pure neuroplasticity and feeding it almost all the data on humanity we can gather as quickly as we can.

What comes out of that is going to be much more human than a human child raised by dogs or put in an isolation box.

Don't get so caught up in the body as what makes us quintessentially human. It's really not.

greyhair• 4 years ago

I think human extinction through human stupidity or hubris is much much much more likely than through an unpredictable path down general AI.

For example, some total whack job of an authoritarian leader is in charge of a sufficient nuclear arsenal and decides to intimidate an adversary by destroying a couple minor cities, and the situation escalates badly. (stupidity)

Or we finally pollute our air and/or water with a persistent substance that either greatly reduces human life span or reproduction rate. (hubris)

I think either of the above is more likely to occur, and I am not commenting on current world events in any way. I think when something bad finally happens, it is going to come completely out of left field. Dr Strangelove style.

And the last of us will be saying "Hmmm, I didn't see that coming".

walleeee• 4 years ago

it happens when autonomous systems optimizing reward functions fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states.

yes, and there is an insight here that I think is often missed in the popular framing of AI x-risk: the autonomous systems we have today (which, defined broadly, need not be entirely or even mostly digital) are just as vulnerable to this

the AGI likely to pose extinction risk in the near term has humans in the loop

less likely to look like Clippy, more likely to look like a catastrophic absence of alignment between loci of agency (social, legal, technical, corporate, political, etc)

rebelos2• 4 years ago

I've always assumed that the trouble will begin when we have a model for 'true' AGI and discover that the constraint "do not harm any human" renders it functionally inert.

kromem5• 4 years ago

The paradoxical idea that AGI is going to still be a monkey paw following simplistic paradigms of operational goals in disastrous ways is hilarious to me every time I see it.

I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it.

I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn.

Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.

Gatsky• 4 years ago

The concern over AI safety seems about right. The unique thing is that anybody cares at all about civilisational negative externalities in a functional and reasonable way. This is quite rare, unprecedented even. Typically humanity just rushes into new endeavours with little concern and leaves future generations to pick up the pieces afterwards (social media, colonialism, processed food, nuclear weapons etc).

IIAOPSW• 4 years ago

2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.

Here's a real generalist question? What's the point of conversation? In the sense that its a "social game", what are the strategy sets on a given turn, and what does it even mean to "win"? Forget about Artificial Bullshiters vs Artificial Idea-what-to-say. How can we even speculate if ABS or "real AI" solves the problem better when we don't really specify the problem space or how to recognize a solution?

In terms of calls and appropriate responses and responses to responses and terminal states, what even is conversation at the protocol level?

liuliu• 4 years ago

For a).1). you basically argued in favor of scalability. We don't know if we are on a training set more or less than a typical human at this point. If I would guess, I think high-bandwidth networked computers can be much more efficient at gathering training set than a single human.

For a).2). you argued in favor of symbolic reasoning. My personal interpretation of that line of thinking is: it helps for us to understand complex thinking machines, but it is not necessarily the building block for the thinking machine.

In the area of AI, there are a lot of opinions, but at the end of the day, builders win the argument. So far, what builders have shown gives me optimistic hope.

nonbirithm• 4 years ago

Personally I'm more worried about what effects sophisticated "dumb" AI will have on human culture and expectations. It would have been hard to imagine we'd be living in this world with our attention assailed by platforms available anywhere when Jobs first held up the iPhone to the world. Just the same, I am not sure that the impending cultural effects from advancements in AI are fully understood at this moment.

For example, I wonder what expectations we will set for ourselves when we pick up a pencil once it becomes common knowledge that production-ready digital art can be created by an AI within minutes. What will people be saying about "old art" in those days? What will we think of people that deliberately choose to disavow AI art?

mdp202113• 4 years ago

worried about what effects sophisticated "dumb" AI will have on human culture and expectations

It is already happening, but it is not a new phenomenon - just the prosecution of the effects of widespread inadequate education.

The undesirable effect namely is, an increase of overvaluing cheap, thin products - a decrease in recognizing actual genius, "substance". For example, some seem to be increasingly contented with "frequentism" as if convinced that language is based on it - while it is of course the opposite - one is supposed to state what is seen, and to state "plausible associations" would have been regarded as a clear sign of malfunction. There seems to be an encouragement to take for "normal" what is in fact deficient.

Some people are brought to believe that some shallow instinct is already the goal and do not know "deep instinct", trained on judgement (active digestion, not passive intake).

The example provided fits:

production-ready digital art

For something to be artistic it has to have foundational justifications - a mockery of art is not art. The Artist chose to draw that line under a number of evaluations that made it a good choice, but an imitator, even in a copy, used the ("frequentist", by the way note) evaluation that the other is an Artist - and there is a world of depth difference between the two.

The difference is trivial, and yet, many are already brought to confuse the shallow mockery and the deep creation.

narrator• 4 years ago

I think the "AGI wants to kill us" meme is just to get us ready for the moment when the authorities unleash the killer robots on us because there's too many of us on the planet. "Whoops, those robots did it all by themselves because it's just inevitable that AGI was going to do that. Haven't you watched a sci-fi movie in the last 50 years?"

kromem5• 4 years ago

(Former emerging tech consultant for ~10% of Fortune 500 here)

(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."

But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.

As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.

So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?

Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.

This is going to continue to occur as long as specialized systems continue to improve and emerge.

And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.

Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."

I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.

Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.

In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.

So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).

(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."

The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.

Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.

There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).

So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?

No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.

We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.

throwaway6532• 4 years ago

In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).

And for this to develop in machines, machines would have to be subject to many mistakes along the way leading to all kinds of outcomes that we hold humans accountable for by fining them, sending them to jail, some of them dying etc. I think that would be so wholly unpalatable to man kind they'd cut that experiment short before it ever reached any sort of scale.

I agree with your conclusion that enough of the rules can't be encoded by us as we don't even know them and for machines to acquire them the traditional way is, I believe, fundamentally disagreeable to humans.

gmadsen• 4 years ago

I'm not so sure its impossible. the 40 year semantic map project Douglas Lenat: Cyc is truly astounding. I think in the next decade we will see really interesting integration of state of the art deep learning with something like Cyc

jasfi• 4 years ago

For those worried about a threats for AGI, this is why such a system must be fully explainable. It's like thinking about a database with no validation at all, if you enter the wrong command you could destroy the integrity of the data. If you have constraints in place you're safer, and of course with SQL you can explain the data and inconsistencies.

My own effort, which focusing on natural language understanding: https://lxagi.com

Abishek_Muthian• 4 years ago

A way to build second/third order reasoning systems...

I've been pondering about this problem for a while now[1], Could we build a collective intelligence through community submitted recipes for second-order decisions for various common activities via generalized schema?

I didn't think about addressing this for AI, But as an aid for us in multi-order thinking. But now that you mention it to be a barrier for AGI, It does make sense.

[1] 'Plan Second-Order, Third-Order consequences': https://needgap.com/problems/263

mvkel• 4 years ago

All great points.

I'd like to offer some perspective as a layman who jumps on "imminent AGI": that's what these AI folks are trying hard to make me think.

It's like the research papers that say there's an "imminent cure to cancer"

karmasimida• 4 years ago

Would this agent able to handle simple elementary mathematics?

If they are using inspiration from Transformer, then it probably won't be able to count.

For that, I don't really feel that enthusiastic about the 'Generalist' claim, maybe they think this is more catchy than just 'Multi-tasking'?

sinenomine9• 4 years ago

A(G)I has become a question of compute economics, for better or for worse. Those with more tightly integrated computational capacity or a good enough logistically sound plan to acquire just enough of it soon enough win, hard.

Should we, the people, watch in awe as our best and brightest financiers chase towards the ultimate prize, the key to all that future entails?

Are those respectable people worthy of the key, and what happens to us in this wild scenario?

evanmoran• 4 years ago

Is this the first reveal of the name Gato? It is the first I’ve heard of it and it definitely sounds like more of a murder bot than a C-3PO :)

I know this is not as important as the AGI question, but I do think the branding matters as much as the attitude of the creators. They seem to be making a generalist agent to see if they can. Gato is a clear name for that: utilitarian and direct. If it was called Sunshine or Gift I suspect the goal would be more helpful to humanity.

drusepth• 4 years ago

Gato, to me, just makes me think "cat", which kind of has a fun ring along "cats on the internet". IMO it sounds more friendly than a robot with a robo-name like C-3PO!

However, I also have a nice robot-named-Gato association[1] from Chrono Trigger. :)

[1] https://i.ytimg.com/vi/ho1TPf2Vj3k/hqdefault.jpg

mrfusion2• 4 years ago

I just can’t tell if this is an incremental improvement or a really huge deal. Any insights?

gcanyon• 4 years ago

Whenever AGI comes up I think of this SMBC comic: https://www.smbc-comics.com/index.php?db=comics&id=2124

simulate-me• 4 years ago

One way I could see this being cool is to drastically cut down the training time of some specific task. DeepMind's model can do a lot of things, but none particularly well. It would be nice if you could start with their model weights and update them to perform well on your specific task with your data. Ideally this process would be cheaper than starting the training from scratch. I also think this is how the human mind develops. There is some intrinsic knowledge that a human starts with, e.g. how to recognize a face and how to grasp, but then training happens over the course of one's life.

kromem5• 4 years ago

It's interesting I so often see the discussion of AI in the context of what a singular model can or can't do.

This seems myopic.

Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for.

And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task.

Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them.

As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.

yarg• 4 years ago

On the topic of Broca's area, families with mutations to the FOXP2 gene are known to have trouble with vocalisations.

https://pubmed.ncbi.nlm.nih.gov/16437554/

There are a number of structures related to specific functionality - facial recognition is similarly hardwired.

Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.

mdp202113• 4 years ago

Our own brains have specialized areas

But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem.

orchestrating ... specialized models

Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.

encore2097• 4 years ago

Getting 406 Not Acceptable - openresty

Downforeveryoneorjust me says its just me -- tried a few different browsers, turned off ublock origin, etc.

edit: Worked when I turned off Wifi on my mobile..

tytung2020• 4 years ago

Do they provide source code or api of the trained model?

DeepMind: A Generalist Agent

Related Stories