2022 Letter

https://zhengdongwang.com/2022/12/28/2022-letter.html

by oli5679 • 4 years ago

143 66 4 years ago

mdorazio• 4 years ago

Do any mainstream AGI researchers believe that GPT-style ML methods will plausibly get us to AGI? I have only a very shallow understanding of the state of the art, but from the outside playing with the tools it seems much more likely that we’ll get to a local maximum far below AGI and require a different approach altogether. I’d love to read some discussion on the topic if anyone has good non-hype/biased fanboy links to share.

ripe• 4 years ago

Sort of the opposite of what you want, but Gary Marcus says we need to cross a few hurdles first:

http://rebooting.ai/

alsodumb• 4 years ago

I am a PhD student working in learning and autonomy space and every researcher I know thinks Gary Marcus is a joke. I'm not saying he doesn't know things, but all I am saying is machine learning at scale is not his area of expertise although he pretends it is. Period. He passes on very generic, obvious statements about the future without any details and when someone does something in that direction he claims 'I told you so!, you should have listened to me in the past!'. Look at the entire chain of discussion between Gary Marcus and Yann LeCun in this thread you'll get a sense of I am talking about: https://twitter.com/ylecun/status/1523305857420824576

Gary Marcus is an academic grifter and to me he is no different than crypto bros who grift non-experts.

hiddencost4• 4 years ago

Seconding reports that Gary Marcus is almost as big a waste of your time as Jurgen Schmidhuber.

Marcus has been writing some variant of exactly the same article multiple times a year for the last 15 years.

readonlybarbie8• 4 years ago

As a professional programmer and a relatively optimistic AGI enthusiast, why would the current ML methods not work, given sufficient CPU/RAM/GPU/latency/bandwidth/storage?

In theory, as long as you can translate your inputs and outputs to an array of floats, a neural network can compute anything. The required number of neurons might not fit into the world's best RAM, and the required number of weights and biases for those neurons might not be quickly calculated by a CPU/GPU however.

azinman2• 4 years ago

Yes but that doesn’t mean you won’t need new architectures or training methods to get there, or data that doesn’t currently exist. We also don’t know how many neurons / layers we’d need, etc.

The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.

jiggawatts• 4 years ago

The current AI approach is like a pure function in programming: no side effects, and given the same input you always get the same output. The “usage” and “training” steps are seperate. There is no episodic memory, especially there is no short term memory.

Biological networks that result in conscious “minds” have a ton of loops and are constantly learning. You can essentially cut yourself off from the outside world in something like a sensory deprivation bath and your mind will continue to operate, talking to itself.

No current popular and successful AI/ML approach can do anything like this.

tmoertel• 4 years ago

One big gap is causal learning. A true general intelligence will have to learn how to intervene in the real world to cause wanted outcomes in novel scenarios. Most current ML models capture only stastical knowledge. They can tell you what interventions have been associated with wanted outcomes in the past. In some situations, replaying these associations seems like genuine causal knowledge, but in novel scenarios this falls short. Even in current day models designed to make causal inferences, say for autonomous driving, the causal structure is more likely to have been built into the models by humans, rather than inferred from observations.

dinkumthinkum• 4 years ago

Well, I would put it back like why would it? When you understand how these things work, does it sound anything like what humans do? When prompted with a question, we do not respond by predicting words that come next based on a gigantic corpus of pre-trained text. As a professional programmer, do you think Human intelligence works like a Turing machine?

isthisthingon992• 4 years ago

Did mainstream AGI researchers predict something like GPT would exist 15 years ago? I would listen to those who did on their opinion.

abecedarius• 4 years ago

There's a 2018 book of interviews of many well-known researchers where they're asked about future prospects: http://book.mfordfuture.com/ (list of interviewees on that page). The actual interview dates weren't specified but don't seem to be earlier than 2017, in my reading. Almost all of them greatly underestimated progress up to now, or refused to say much. (I'm hedging a little bit because it's a year since I read it, and memory is fuzzy.)

Shane Legg of DeepMind wrote a blog post at the opening of the 2010s where he stuck his neck out to predict AGI with a time distribution peaking around 2030. He thought the major development would be in reinforcement learning, rather than the self-supervised GPT stuff.

Symmetry• 4 years ago

We still seem to be missing an equivalent of explicit memory formation, serializing digested perceptions into working then short term and long term memory. The however many thousand tokens in a GPT's buffer can span a much larger span of time than the second's worth of sense impressions your brain can hold without consciousness[1] and memory getting involved but the principle seems to be the same.

This isn't to say that there wouldn't be some simple hack to allow memory formation in chat agents, just that there's at least one advance we need besides simple scale.

[1] As in not subliminal, not anything to do with philosophical notions of qualia.

lumost• 4 years ago

I haven't seen a good write-up on this, however it appears that large Transformer architectures are directly learning a memory representation in their parameters. ChatGPT can easily return facts, and approximate summaries of facts. Unfortunately ChatGPT is unable to differentiate what it knows, vs. what it thinks it knows. I'd love to see a deep dive into how these models are representing such knowledge.

amelius2• 4 years ago

Also missing seems to be generalization. Train a robot to kick a white ball, and it will not understand how to kick a green ball.

hiddencost4• 4 years ago

I don't think anyone serious thinks or talks in terms of AGI. The feverishly simplistic idea of the singularity is quite silly.

Most notably, neural networks alone will not reach any kind of AGI.

Start adding the capacity to read from massive knowledge stores, and a place to keep long term information (i.e., memory, probably also in a database), plus a feedback loop for the model to learn and improve? Plus the ability to call APIs? Now you're talking. I think all of those pieces are close to doable right now, maybe with a latency of 5s. If one of the big players puts that in place in a way that is well measured and they can iterate on, I think we'll start to see some really incredible advances.

hiddencost4• 4 years ago

One really exciting place things might improve is in data cleaning. Right now preprocessing your data and putting it in a format that can be learned efficiently and without bias is a huge pain // risk. This next generation is allowing us to largely ignore a lot of that work.

Similarly, transfer learning is finally good.

And the models are generalist, few shot learners.

As a consequence, individuals with minimal expertise can set up a world class system to solve niche problems. That's really exciting and it's going to get easier.

localhost2• 4 years ago

The gpt_index project looks very promising in this area.

"At its core, GPT Index is about:

1. loading in external data (@NotionHQ, @Slack, .txt, etc.) 2. Building indices over that data 3. Inputting a prompt -> getting an output!"

https://twitter.com/jerryjliu0/status/1608632335695745024

0x008• 4 years ago

Actually they don't. You can listen to the podcasts by Lex Fridman with Yan Le Cun or Andrej Karpathy regarding that topic. But basically what Le Cun is saying is that the information density in text is not high enough in order to learn a realistic representation of the world from it.

mikepurvis• 4 years ago

I'm completely a bystander, but I feel like one flag for me with current approaches is the ongoing separation between training and runtime. Robotics has been through a similar thing where you have one program that does SLAM while you teleop the robot, and you use that to map your environment, then afterward shut it down and pass the static map into a separate localization + navigation stack.

Just as robots have had to graduate to the world of continuous SLAM, navigating while building and constantly updating a map, I feel like there's a big missing piece in current AI for a system that can simultaneously act and learn, that can reflect on gaps in its own knowledge, and express curiosity in order to facilitate learning— that can ask a question out of a desire to know rather than as a party trick.

ben_w7• 4 years ago

I think that depends on which definition of AGI you prefer. It (GPT-3) knows more than I do about most topics (though I still beat it at the stuff I'm best at), so I'd say it's got the A and the G covered, and it's "only" at the level of a university student at the stuff I've seen it tested on, so it's "pretty intelligent" in the way toy drone submarines are "pretty good swimmers".

It's not as fast a learner (efficient with samples) as humans are; it doesn't continuously learn from interactions with others like we do; and it's certainly not superhuman at everything (or probably anything other than breadth of knowledge)…

…but Yudkowsky recently criticised Musk for taking that too far and limiting the definition of "AGI" to what is normally meant by "ASI" (Yudkowsky was also saying that no, GPT isn't AGI): https://twitter.com/ESYudkowsky/status/1600362288149856256?c...

mik19986• 4 years ago

Last time I checked ChatGPT was unable to solve basic elementary school math problems.

6gvONxR4sf7o• 4 years ago

That kind of ML model is pretty general. Pretraining a big model has been extended to multi-modal environments. People are training them with RL to take actions. People are applying other generative techniques to them, and all sorts of other stuff. If you just look at it as 'predict the next word token,' then it's pretty limiting, but people have already gone way beyond that. TFA talks about some interesting directions people are taking them.

A more general form of your question is whether we can get to AGI with just incremental steps from where we are today, rather than step-change way-out-of-left-field kinds of ideas. People are split on that. Personally, I think that incremental changes from today's methods are sufficient with better hardware and data, but Big New Ideas could certainly speed up progress.

AnimalMuppet• 4 years ago

Extremely cynical take: GPT-style ML will get us to a place that we can fool people into thinking that AGI is just around the bend, or maybe one more breakthrough away. It gives people hope, but is not a realistic foundation.

But watch for lots of breathless "look how close we are!" messages.

The other thing GPT is good for is polluting our info-sphere with stuff that sounds confident, but may be anywhere from dead right to slightly off to completely wrong. Having automated means of producing high volumes of fine-sounding nonsense is not the path to anywhere good.

logicallee2• 4 years ago

From my experience working with ChatGPT, I believe that it exhibits many of the characteristics of AGI. For example, it can correctly interpret and parse novel requests, complete tasks that are virtually impossible for a machine to accomplish (example below), perform at the level of an average high school student in most areas, demonstrate judgment and opinion (go ahead, ask it to give you an opinion about something you propose to it), and assemble plans for novel situations. However, ChatGPT still has limitations, such as difficulty with numeracy, handling individual characters, and visual tasks. It's also worth noting that ChatGPT's mistakes sometimes sound like those of a child, and it can sometimes produce code that needs minor changes to work. Despite these obvious shortcomings, it passes my personal standard for the first demonstration of limited AGI.

[above is based on chatgpt's summary of my long draft here: https://hastebin.com/raw/muzuvodupu - I also added the last line above and the parentheticals.]

ilaksh2• 4 years ago

As far as the supervisor/worker idea, it can act as both. You can just say "outline the steps for this" and then "implement step 1 above", "implement step 2". It does much better than when you try to get it to fulfill an entire complex task in one shot.

I have been testing some things with text-davinci-003 which is not the same as ChatGPT but quite similar. https://aidev.codes/u/runvnc/flappy/edit

tim333• 4 years ago

AGI will probably come from something like GPT-style ML methods plus something else IMHO. They won't get there on their own but the research will be useful.

fmajid• 4 years ago

Yes, Dan Wang's letters are outstanding.

laidoffamazon• 4 years ago

Realizing now that the author went to Yale.

Are there any notable figures in the space that didn’t go or get into top schools? Feels like it’s becoming an elite echo chamber.

kid642• 4 years ago

The 286 was featured in the IBM AT, not XT.

LBJsPNS2• 4 years ago

There was also an XT 286. 286 processor, 8-bit bus from the PC as opposed to the AT's 16-bit bus.

https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmw...

kid642• 4 years ago

I'll be darned. I stand corrected!

lordofmoria2• 4 years ago

In 2008, I was taking COS 217 at Princeton - our final project was to write a program to play Othello competitively. We were told to use alpha-beta-pruning for grading, but that there would be a tournament at the end of class, so we could tweak our programs for that (the assignment was insanely hard, so of course no one had time to tweak).

To my surprise, I won the tournament, despite having done the bare implementation with no tweaks! I didn’t get a perfect grade on the assignment though - apparently I had miss implemented the queue for the alpha-beta search, and had completely reversed the queue ordering.

Of course, searching “in reverse” through the alpha-beta tree turned out to be the difference that won me the tournament. It was rather humiliating, but a good lesson on sheer luck’s role in academic approaches to AI.

dmurray• 4 years ago

Isn't searching the right way round extremely important? If you get the ordering right, a search to depth N takes time O(2^N), if you get it in reverse, O(2^2N). Perhaps if there were no time constraints, doing something slightly different was enough to beat the other competitors who cancelled each other out.

lordofmoria2• 4 years ago

Sadly the mistake was even more basic than you’re likely thinking - as I recall, min-max is necessarily breath-first, and I simply pushed onto the queue from the wrong side (think: append instead of push), so the queue was backwards, and breadth first traversed from right to left. Same time complexity, but slightly different paths reached within the same time constraints!

chairmanwow1• 4 years ago

This is a great read, but I keep getting pulled astray by the sheer volume of interesting links that the author has embedded in the text. I'm less than a third of the way through and have just finished my 3rd rabbit hole exploration.

Scubabear68• 4 years ago

I think Heinlein hit an essence in The Moon is a Harsh Mistress for AGI.

It needs many inputs and many outputs into the real world. A true AGI sees the reaction of the world to its own actions. That is the root of sentience.

random314• 4 years ago

The font on the blog is beautiful

2022 Letter

Related Stories