Thousands of AI Authors on the Future of AI

by treebrained • 3 years ago

85 112 3 years ago

bbor3• 3 years ago

Very interesting, especially the huge jump forward in the first figure and a possible majority of AI researchers giving >10% to the Human Extinction outcome.

To AI skeptics bristling at these numbers, I’ve got a potentially controversial question: what’s the difference between this and the scientific consensus on Climate Change? Why heed the latter and not the former?

lainga• 3 years ago

A climate forcing has a physical effect on the Earth system that you can model with primitive equations. It is not a social or economic problem (although removing the forcing is).

You might as well roll a ball down an incline and then ask me whether Keynes was right.

HPMOR• 3 years ago

Wait I gotta defend my boy Keynes here. His predictions have been as nearly as well validated as predicting the outcome of a ball rolling down a plank. Just reading the first part of the General Theory correctly predicted the labor strikes in 2023. Keynes’ very clear predictions continue to hold up under empirical observation.

bbor3• 3 years ago

Ha, well said, point taken. I’d say AI risk is also a technology problem, but without quantifiable models for the relevant risks, it stops sounding like science and starts being interpreted as philosophy. Which is pretty fair.

If I remember an article from a few days ago correctly, this would make the AI threat an “uncertain” one, rather than merely “risky” like climate change (we know what might happen, we just need to figure out how likely it is).

EDIT: Disregarding the fact that in that article, climate change was actually the example of a quintessentially uncertain problem… makes me chuckle. A lesson on relative uncertainty

staunton2• 3 years ago

I would think any scenario where humans actually go extinct (as opposed to just civilization collapsing and the population plummeting, which would be terrible enough) has to involve a lot of social and economic modeling...

michael_nielsen• 3 years ago

We have extremely detailed and well-tested models of climate. It's worth reading the IPCC report - it's extremely interesting, and quite accessible. I was somewhat skeptical of climate work before I began reading, but I spent hundreds of hours understanding it, and was quite impressed by the depth of the work. By contrast, our models of future AI are very weak. Something like the scaling laws paper or the Chinchilla paper are far less convincing than the best climate work. And arguments like those in Nick Bostrom or Stuart Russell's books are much more conjectural and qualitative (& less well-tested) than the climate argument

I say this as someone who written several pieces about xrisk from AI, and who is concerned. The models and reasoning are simply not nearly as detailed or well-tested as in the case of climate.

bbor3• 3 years ago

Amazing response, thanks for taking the time - concise, clear, and I don’t think I’ll be using that comparison again because of you. I see now how much more convincing mathematical models are than philosophical arguments in this context, and why that allows modern climate-change-believing scientists to dismiss this (potential, very weak, uncertain) cogsci consensus.

In this light, widespread acknowledgement of xrisk will only come once we have a statistical model that shows it will. And at that point, it seems like it would be too late… Perhaps “Intelligence Explosion Modeling” should be a new sub-field under “AI Safety & Alignment” — a grim but useful line of work.

FAKE_EDIT: In fact, after looking it up, it sorta is! After a few minutes skimming I recommend Intelligence Explosion Microeconomics (Yudkowsky 2013>) to anyone interested in the above. On the pile of to-read lit it goes…

kranke155• 3 years ago

Because the AI human extinction idea is entirely conjecture while climate change is just a progression on current models.

What’s the progression that leads to AI human extinction ?

blamestross• 3 years ago

Profit motive

chasd00• 3 years ago

There's also the possibility of a AGI being benevolent or, what i find more likely, suicidal.

Vecr3• 3 years ago

Just hope it won't try to take us with it (see Dark Star's "smart bomb").

staunton2• 3 years ago

I think human extinction due to climate change is extremely unlikely. However, civilization collapsing is bad enough. We can't be certain whether or not that will actually happen but we do know it will if we do nothing. We even have a pretty good idea when, that's not too late yet, and we have an actionable scientific consensus about what to do about it.

In many ways AI risk looks like the opposite. It might actually cause extinction but we have no idea how likely that is and neither do we have any idea how likely any bad not-quite-extinction outcome is. The outcome might even be very positive. We have no idea when anything will happen and the only realistic plan that's sure to avoid the bad outcome is to stop building AI, which also means we don't get the potential good outcome, and there's no scientific consensus about that (or anything else) being a good plan because it's almost impossible to gather concrete empirical evidence about the risk. By the time such evidence is available, it might be too late (this could also have happened with climate change, we got lucky there...)

idopmstuff3• 3 years ago

If science continues undisrupted, the chance of unaided machines outperforming humans in every possible task was estimated at 10% by 2027, and 50% by 2047.

Maybe I'm overly optimistic (or pessimistic depending on your point of view, I suppose), but 50% by 2047 seems low to me. That just feels like an eternity of development, and even if we maintain the current pace (let alone see it accelerate as AI contributes more to its own development), it's difficult for me to imagine what humans will still be better able to do than AI in over a decade.

I do wonder if the question is ambiguously phrased and some people interpreted it as pure AI (i.e. just bits) while others answered it with the assumption that you'd also have to have the sort of bipedal robot enabled with AI that would allow it to take on all the manual tasks humans do.

sdenton4• 3 years ago

Yeah, it really comes down to the question of how we advance on just-bits vs constrained-environment robotics vs open-domain robotics...

Some interesting work here on using LLMs to improve on open-domain robotics: https://arstechnica.com/information-technology/2023/03/embod...

bcrosby95• 3 years ago

What is the "current pace". Last year? Last 5 years? Last 20 years?

If you mean the last year, is that pace maintainable?

idopmstuff3• 3 years ago

If you mean the last year, is that pace maintainable?

That is the question, though I'd turn it around on you - over the course of human history, the speed of progress has been ever-increasing. As AI develops, it is itself a new tool that should increase the speed of progress. Shouldn't our base case be the assumption that progress will speed up, rather than to question whether it's maintainable?

Workaccount2• 3 years ago

If you gave print outs of discussions with GPT-4 to AI researchers 5 years ago, they would have told you conversation like that is 10 or 20 years out.

drtz• 3 years ago

I'm of the opposite opinion. I think there's some Dunning-Kruger-like effect at play on a macro scale and it's causing researchers to feel like they're closer than they are because they're in uncharted territory and can't see the complexity of what they're trying to build.

Or maybe I'm just jaded after a couple decades of consistently underbidding engineering and software projects :)

edit: Fix typo

mistrial92• 3 years ago

a single number as percentage is not useful here.. Intense video games ? of course.. plumbing professionals in a city ? not even close.. etc

jelsisi• 3 years ago

As a 20 something this makes me so excited to be alive.

teddyh• 3 years ago

“I've come up with a set of rules that describe our reactions to technologies:

1. Anything that is in the world when you’re born is normal and ordinary and is just a natural part of the way the world works.

2. Anything that's invented between when you’re fifteen and thirty-five is new and exciting and revolutionary and you can probably get a career in it.

3. Anything invented after you're thirty-five is against the natural order of things.”

― Douglas Adams, The Salmon of Doubt

vladms• 3 years ago

That's really a characteristic of being 20 (with the "this" being different things of different people). Good for everybody that manages to keep the feeling later as well, but it is definitely easier at 20...

endisneigh9• 3 years ago

Maybe I'm too pessimistic, but I doubt we will have AGI by even 2100. I define AGI as the ability for an intelligence that is not human to do anything any human has ever done or will do with technology that does not include itself* (AGI).

* It also goes without saying that by this definition I mean to say that humanity will no longer be able to meaningfully help in any qualitative way with respect to intellectual tasks (e.g. AGI > human; AGI > human + computer; AGI > human + internet; AGI > human + LLM).

Fundamentally I believe AGI will never happen without a body. I believe intelligence requires constraints and the ultimate constraint is life. Some omniscient immortal thing seems neat, but I doubt it'll be as smart since it lacks any constraints to drive it to growth.

Nevermark2• 3 years ago

If we consider OpenAI itself, a hybrid corporation/AI system, it's constraints are obvious.

It needs vast resources to operate. As the competition in AI heats up, it will continually have to create new levels of value to survive.

Not making any predictions about OpenAI, except that as its machines get smarter, they will also get more explicitly focused on its survival.

(As apposed to the implicit contribution of AI to its creation of value today. The AI is in a passive role for the time being.)

murderfs• 3 years ago

I define AGI as the ability for an intelligence that is not human to do anything any human has ever done or will do with technology that does not include itself* (AGI).

That bar is insane. By that logic, humans aren't intelligent.

endisneigh9• 3 years ago

What do you mean? By that same logic humans definitionally already have done everything they can or will do with technology.

I believe AGI must be definitionally superior. Anything else and you could argue it’s existed for a while, e.g. computers have been superior at adding numbers basically their entire existence. Even with reasoning, computers have been better for a while. Language models have allowed for that reasoning to be specified in English, but you could’ve easily written a formally verified program in the 90s that exhibits better reasoning in the form of correctness for discrete tasks.

Even with game playing Go, and Chess, games that require moderate to high planning skills are all but solves with computers, but I don’t consider them AGI.

I would not consider N entities that can each beat humanity in the Y tasks humans are capable of to be AGI, unless some system X is capable of picking N for Y as necessary without explicit prompting. It would need to be a single system. That being said I could see one disagreeing haha.

I am curious if anyone has different definition of AGI that cannot already be met now.

novagameco3• 3 years ago

I'm optimistic in that I hope we don't have AGI by 2100 because it sounds like a truly dystopian future even in the best case scenario

ehsanu1• 3 years ago

What kind of best case are you imagining? I don't quite understand why the very best case would be dystopian.

albertzeyer• 3 years ago

You say that a single AGI model should be as powerful as doing everything what the whole humanity has done in the last 100k years or so?

Or a group of millions of such AGI instances in a similar time frame?

endisneigh9• 3 years ago

No, not a single model. A single system. Based off of nothing I expect AGI to basically be implemented like MoE.

alanbernstein• 3 years ago

Fundamentally I believe AGI will never happen without a body

I'm inclined to believe this as well, but rather than "it won't happen", I take it to mean that AI and robotics just need to unify. That's already starting to happen.

bufferoverflow2• 3 years ago

That's an unreasonable metric for AGI.

You're basically requiring AGI to be smarter/better than the smartest/best humans in every single field.

What you're describing is ASI.

If we have AGI that is on the level of an average human (which is pretty dumb), it's already very useful. That gives you robotic paradise where robots do ALL mundane tasks.

endisneigh9• 3 years ago

What is your definition for AGI that isn't already met? Computers have already been superior to average humans in a variety of fields since the 90s. If we consider intelligence as the ability to acquire knowledge, then any "AGI" will be "ASI" in short order, therefore I make no distinction between the two.

breck• 3 years ago

I doubt we will have AGI by even 2100...Fundamentally I believe AGI will never happen without a body.

I think this is very plausible--that AI won't really be AGI until it has a way to physically grow free from the umbilical chord that is the chip fab supply chain.

So it might take Brainoids/Brain-on-chip technology to get a lot more advanced before that happens. However, if there are some breakthroughs in that tech, so that a digital AI could interact with in vitro tissue, utilize it, and grow it, it seems like the takeoff could be really fast.

paxys2• 3 years ago

AGI doesn't have to mean superintelligence/singularity (which seems to be what you are describing).

endisneigh9• 3 years ago

What is your definition for AGI that isn't already met?

golol3• 3 years ago

Why define AGI like that? General intelligence is supposed to be something like human intelligence. You are talking about ASI.

endisneigh9• 3 years ago

I'm curious to hear your definition of AGI that hasn't already been met, given computers have been superior to humans at a large variety of tasks since the 90s.

abeppu2• 3 years ago

There's a lot of cool work being done on embodied intelligence -- what makes you think that 76 years wouldn't be enough to create an embodied agent with relevant in-built constraints?

light_hue_12• 3 years ago

I got this survey; for the record I didn't respond.

I don't think their results are meaningful at all.

Asking random AI researchers about automating a field they have no idea about means nothing. What do I know about the job of a surgeon? My opinion on how current models can automate a job I don't understand is worthless.

Asking random AI researchers about automation outside of their area of expertise is also worthless. A computer vision expert has no idea what the state of the art in grasping is. So what does their opinion on installing wiring in a house count for? Nothing.

Even abstract tasks like translation. If you aren't an NLP researcher who has dealt with translation you have no idea how you even measure how good a translated document is, so why are you being asked when translation will be "fluent"? You're asking a clueless person a question they literally cannot even understand.

This is a survey of AI hype, not any indication of what the future holds.

Their results are also highly biased. Most senior researchers aren't going to waste their time filling this out (90% of people did not fill it out). They almost certainly got very junior people and those with an axe to grind. Many of the respondents also have a conflict of interest, they run AI startups. Of course they want as much hype as possible.

This is not a survey of what the average AI researcher thinks.

treebrained• 3 years ago

Thank you for this comment. It is great to hear an inside take.

Idle curiosity, but what NLP tools evaluate translation quality better than a person? I was under the (perhaps mistaken) impression that NLP tools would be designed to approximate human intuition on this.

The survey does address the points above a bit. Per Section 5.2.2 and Appendix D, the survey had a response rate of 15% overall and of ~10% among people with over 1000 citations. Respondents who had given "when HLMI [more or less AGI] will be developed" or "impacts of smarter-than-human machines" a "great deal" of thought prior to the survey were 7.6% and 10.3%, respectively. Appendix D indicates that they saw no large differences between industry and academic respondents besides response rate, which was much lower for people in industry.

light_hue_12• 3 years ago

Idle curiosity, but what NLP tools evaluate translation quality better than a person? I was under the (perhaps mistaken) impression that NLP tools would be designed to approximate human intuition on this.

This is a long story. But, your take on this question is what the average person who responded to that survey knows. And it shows you really how little the results mean. Here are some minutia that really matter:

1. Even if you measure quality with people in the loop. What do you ask people? Here's a passage in English, one in French, do you agree? Rate it out of 10? Turns out people aren't calibrated at all to give reasonable ratings, you get basically junk results if you run this experiment.

2. You can ask people to do head to head experiments. Do you like translation A more than translation B? But.. what criteria should they use? Is accuracy what matters most? Is it how would they translate? Is it how well A or B reads? Is it how well it represents the form of the source? Or the ideas of the source?

3. Are we measuring sentences? Paragraphs? Pages? 3 word sentences "give me grool" are pretty easy. 3 page translations get tricky. Now you want to represent something about the style of the writer. Or to realize that they're holding something back. For example, it can be really obvious in French that I'm holding back someone's gender, but not obvious at all in English. What about customs? Taboos? Do we even measure 3 pages worth of translation in our NLP corpora? The respondents have no idea.

4. There are even domain-specific questions about translations. Do you know how to evaluate English to French in the context of a contract? One that goes from common law to civil law? No way. You need to translate ideas now, not just words. How about medical translation? Most translation work is highly technical like this.

I could go on. Mostly we don't even measure minutia about translations or domain-specific translation in our NLP benchmarks because the tools aren't good enough for that. Nor do we measure 5 page translations for their fidelity.

We actually mostly don't measure translations using humans at all! We collect translations from humans and then we compare machine translations to human translations after the fact, with something called parallel corpora (the historical example is the Hansard corpus; which is the proceedings of the Canadian parliament that are manually translated in English and French; the EU has also been a boon for translation research).

I'm scratching the surface here. Translation is a really complicated topic. My favourite book related to this is the Dictionary of Untranslatables https://press.princeton.edu/books/hardcover/9780691138701/di... Not something you'd read end-to-end but a really fun reference to dip into once in a while.

If someone who knows about these issues wants to say that there will be human-level translation AI in 10 years, ok, fine I'm willing to buy that. But if someone who is ignorant of all of this is trying to tell me that there will be human level AI for translation in 10 years, eh, they just don't know what they're talking about. I am by the way a translation visitor, I've published in the area, but I'm not an expert at all, I don't even trust my opinion on the subject of when it will be automated.

About biases, I saw appendix A and D.

Seniority doesn't mean >1000 citations. There are master's students with 1000 citations in junk journals who happened to get a paper in a better venue. Number of citations is not an indication of anything.

The way they count academia vs industry is meaningless. There are plenty of people who have an affiliation to a university but are primarily at a startup. There are plenty of people who are minor coauthors on a paper, or even faculty who are mostly interested in making money off of the AI hype. There are plenty of people who graduated 3 years ago, this is a wrap-up of their work, they counted as academic in the survey, but now they're in industry. etc.

sveme• 3 years ago

Does anyone know potential causal chains that bring about the extinction of mankind through AI? Obviously aware of terminator, but what other chains would be possible?

kristianc• 3 years ago

I'm fascinated by this as well. There's a lot of conjecture around what if, but I'm yet to really hear much about the how.

Nevermark2• 3 years ago

Machines won't need the biosphere to survive.

If they accelerate the burning of fossil fuels, extract and process minerals on land and in the ocean without concern for pollution, replace large areas of the natural world with solar panels, etc., the world could rapidly become hostile for large creatures.

An ocean die out as a result of massive deep sea mining would be particularly devastating. It's very hard to contain pollution in the ocean.

Same for lakes. And without clean water things will get bad everywhere.

Ramping up the frequency of space launches a few orders of magnitude into the solar system for further resources could heavily pollute the atmosphere.

Microbes might be fine, and be able to evolve to changes, for much longer.

Vecr3• 3 years ago

I'm going to take that to mean "P(every last human dead) > 0.5" because I can't model situations like that very well, but if for some reason (see Thucydides Trap for one theory, instrumental convergence for another) the AI system thinks the existence of humans is a problem for its risk management, it would probably want to kill them. "All processes that are stable we shall predict. All processes that are unstable we shall control." Since humans are an unstable process, and the easiest form of human to control is a corpse, it would be rational for an AI system that wants to improve its prediction of the future to kill all humans. It could plausibly do so with a series of bioengeneered pathogens, possibly starting with viruses to destroy civilization then moving on to bacteria dropped into water sources to clean up the survivors (as they don't have treated drinking water anymore due to civilization collapsing). Don't even try with an off switch, if no human is alive to trigger it, it can't be triggered, and dead man's switches can be subverted. If it thinks you hid the off switch it might try to kill everyone even if the switch does not exist. In that situation you can't farm, because farms can be seen from space (and an ASI is a better analyst than any spy agency could be), you can't hunt because all the animals are covered inside and out with special anti-human bacteria, natural water sources are also fully infected.

notahacker• 3 years ago

If the AGI - which is for some reason always imagined as a singular entity - thinks humans are unpredictable and risky now, just imagine the unpredictability and risk involved in trying to kill all seven billion of us whilst keeping the electricity supply on...

jetrink• 3 years ago

To borrow a phrase from Microsoft's history, "Embrace, Extend, Extinguish." AI proves to be incredibly useful and we welcome it like we welcomed the internet. It becomes deeply embedded in our lives and eventually in our bodies. One day, a generation is born that never experiences a thought that is not augmented by AI. Sometime later a generation is born that is more AI than human. Sometime later, there are no humans.

daxfohl• 3 years ago

People wouldn't even get a vaccine injection because there were supposedly Bill Gates's microchips in them. Now you expect they'll be flocking to get these because they have Bill Gates's microchips in them?

Actually, I could see that happening.

Maybe it's time to give AGI a chance to run things anyway and see if it can do any better. Certainly it isn't a very high bar.

__loam• 3 years ago

One potential line is a general purpose ai like chat gpt that can give instructions on how to produce genetically engineered viral weapons, for example. I find this improbable, but it's possible that a future LLM (or whatever) gets released that has this capability but isn't know to have that capability. Then you might have a bunch of independent actors making novel contagions in their garage.

That would still require a lot of equipment potentially but it's there.

Another possibility would be some kind of rogue agent scenario where the program and hide and distribute itself on many machines, and interact with people to get them to do bad things or give it money. I think someone already demonstrated one of the LLMs doing some kind of social engineering attack somewhere and getting the support agent to let them in. Not hard to imagine some kind of government funded weapon that scales up that kind of attack. Imagine whole social movements, terrorist groups, or religious cults run by an autonomous agent.

walkhour• 3 years ago

This argument gives a 35% chance of AI "taking over" (granted this does not mean extinction) this century: https://www.foxy-scout.com/wwotf-review/#underestimating-ris.... The argument consists of 6 steps, assigning probabilities to each step, and multiplying the probabilities.

mnky9800n• 3 years ago

I often think about this from a standpoint of curiosity. I am simply curious about how the universe works, how information is distributed across it, how computers use it, and how this all connects through physics. If I'm soon to be greeted by an AI friend who shares my interests then that's a welcome addition to my friend and colleagues circle. I'm not really sure why I wouldn't continue pursuing my interests simply because there's someone better at doing it then me. There are many people that I know better at this than me. Why not add a robot to the mix?

jjcm2• 3 years ago

A really simple approach we took while I was working on a research team at Microsoft for predicting when AGI would land was simply estimating at what point can we run a full simulation of all of the chemical processes and synapses inside a human brain.

The approach was tremendously simple and totally naive, but it was still interesting. At the time a supercomputer could simulate the full brain of a flatworm. We then simply applied a Moore's law-esque approach of assuming simulation capacity can double every 1.5-2 years (I forget the time period we used), and mapped out different animals that we had the capability to simulate on each date. We showed years for a field mouse, a corvid, a chimp, and eventually a human brain. The date we landed on was 2047.

There are so many things wrong with that approach I can't even count, but I'd be kinda smitten if it ended up being correct.

tayo42• 3 years ago

Is there something to read about simulating a worm brain. Neurons aren't just simply on and off? They grow and adapt physically along with their chemical signals. Curious how a computer accounts for all of that.

jncraton• 3 years ago

You might be interested in OpenWorm:

https://openworm.org/

This paper might be helpful for understanding the nervous system in particular:

https://royalsocietypublishing.org/doi/10.1098/rstb.2017.037...

Keloo• 3 years ago

How does it compare to the progress? Is the progress faster/slower?

Any links to read?

shpongled4• 3 years ago

To be pedantic, I would argue that we aren't even close to being able to simulate the full brain of a flatworm on a supercomputer at anything deeper than a simple representation of neurons.

We can't even simulate all of the chemical processes inside a single cell. We don't even know all of the chemical processes. We don't know the function of most proteins.

minroot• 3 years ago

Does AGI must needs to be brain-like?

consumer451• 3 years ago

We can't even simulate all of the chemical processes inside a single cell. We don't even know all of the chemical processes. We don't know the function of most proteins.

Brain > Cell > Molecules(DNA and otherwise) > Atoms > Sub-atomic particles...

Potentially dumb question, but how deeply do we need to understand the underlying components to simulate a flatworm brain?

throwup2385• 3 years ago

The vast majority of the chemical processes in a single cell are concerned with maintaining homeostasis for that cell - just keeping it alive, well fed with ATP, and repairing the cell walls. We don't need to simulate them.

gary_02• 3 years ago

It depends on what kind of simulation you're trying to run, though. You don't need to perfectly model the physically moving heads and magnetic oscillations of a hard drive to emulate an old PC; it may be enough to just store the bytes.

I suspect if you just want an automaton that provides the utility of a human brain, we'll be fine just using statistical approximations based on what we see biological neurons doing. The utility of LLMs so far has moved the needle in that direction for sure, although there's still enough we don't know about cognition that we could still hit a surprise brick wall when we start trying to build GPT-6 or whatever. But even so, a prediction of 2047 for that kind of AGI is plausible (ironically, any semblance of Moore's Law probably won't last until then).

On the other hand, if you want to model a particular human brain... well, then things get extremely hairy scientifically, philosophically, and ethically.

jjcm2• 3 years ago

I very likely was incorrect about the chemical processes, so thank you for clarifying. This is remembering work for a decade ago, so I'm almost certainly wrong about some of the details.

throwup2385• 3 years ago

Another approach: the adult human brain has 100 (+- 20) billion or 10^11 neurons. Each neuron has 10^3 synapses and each synapse has 10^2 ion channels, amounts to 10^16 total channels. Assuming 10 parameters is enough to represent each channel (unlikely), that's about 10^17 (100 quadrillion) total parameters. Compare that to GPT4 which is rumored to be about 1.7*10^12 parameters on 8x 80GB A100s.

log(10^17/10^12)/log(2) = 16.61 so assuming 1.5 years per doubling, that'll be another 24.9 years - December, 2048 - before 8x X100s can simulate the human brain.

WhitneyLand• 3 years ago

And then how long until it runs on 20 watts of power? ;)

redavni• 3 years ago

December, 2048

So it is not unreasonable to expect I can have an Ana de Armas AI in 2049?

I hope you AI people are better than the flying car people.

daveguy• 3 years ago

I assume simulation capacity takes into account the data bandwidth of the processing systems. It seems we are always an orders of magnitude or two behind in bytes/words per second to feed simulations compared to raw flops. When you consider there are multiple orders of magnitude more synapses between neurons than neurons (not to mention other cell types we are only beginning to understand) -- bandwidth limitations seem to put estimates about 10-15 years past computation estimates. By my napkin math, accounting for bandwidth limitations, we will get single-human-intelligence hardware capabilities 2053-2063. Whether or not we've figured out the algorithms by then is any guess. Maybe algorithm advances will reduce hardware needs, but I doubt it because computational complexity to solve hard problems is often a matter of getting all the bits to the processor to perform all the comparisons necessary. However, the massive parallelism of the brain is a point of optimism.

ShamelessC2• 3 years ago

At the time a supercomputer could simulate the full brain of a flatworm.

Citation needed?

ShamelessC2• 3 years ago

Just going to follow up and say “I don’t think that statement is even remotely true now, much less back then”. We haven’t accurately simulated any life forms. Failure to simulate C. Elegans is notable.

reality71• 3 years ago

Your approach will eventually work, no doubt about it, but the question is whether the amount of energy the computer uses to complete a task is less than the energy the equivalent conglomeration of humans use to complete a task.

It seems clear at this point that although computers can be made to model physical systems to great degree, this is not the area where they naturally excel. Think of modeling the temperature of a room, you could try and recreate the physically accurate simulation of every particle and its velocity. We could then create better software to model the particles on ever more powerful and specific hardware to model bigger and bigger rooms.

Just like how thermodynamics might make more sense to model statistically, I think intelligence is not best modeled at the synapse layer.

I think the much more interesting question is what would the equivalent of a worm brain be for a digital intelligence?

fasterik• 3 years ago

It's interesting data on what AI researchers think, but why should we think that AI researchers are going to have the most accurate predictions about the future of AI? The skills that make someone a good AI researcher aren't necessarily the same skills that make someone a good forecaster. Also, the people most likely to have biased views on the subject are people working within the field.

mtdefelice• 3 years ago

I seriously feel crazy when reading stuff like this. At best this seems to describe the statistics of the consensus of the researchers - lacking any of the social, legal, political, and economic drivers of progress for each prediction. Also, how are these "tasks" or "jobs" actually defined (and how is the believability of each respondent considered when aggregating predictions as it relates to each)? What am I missing?

mmaunder2• 3 years ago

I think history has shown us that we tend to underestimate the rate of technological progress and it's rate of acceleration.

It's tempting to look at Moore's law and use say the development of the 8080, z-80 and 6502 in 1975 as an epoch. But it's hard to use that to get a visceral sense of how much things changed. I think RAM - in other words, available memory - may be more helpful, and it does relate in a distant way with model size and available GPU memory.

So the question is, if we surveyed a group of devs, engineers and computer scientists in 1975 and asked them to extrapolate and predict available RAM a few decades out, how well would their predictions map to reality?

In 1975 the Altair 8800 microcomputer with the 8080 processor had 8K of memory for the high end kit (4096 words).

8 years later, in 1983 the Apple IIe (which I learned to program on) had 64K RAM as standard, or 8 times the RAM.

13 years later in 1996, 16 to 32 MB was fairly commonplace in desktop PCs. That's 32,768K which is 4096 times the 8K available 21 years earlier.

30 years later in 2005, it wasn't unusual to find 1GB of RAM or 1,048,576K or 131,072 times 8K from 30 years earlier.

Is it realistic to expect a 1975 programmer, hardware engineer or computer scientist to predict that available memory in a desktop machine will be over 100,000 times greater 30 years in the future? We're not even taking into account moving from byte oriented CPUs to 32bit CPUs, or memory bandwidth.

2054 is 30 years in the future. It's going to fly by. I think given the unbelievable rate of change we've seen in the past, and how it accelerates, any prediction today from the smartest and most forward thinking people in AI will vastly underestimate what 2054 will look like.

Edit: 32bit CPU's not 64. Typo.

mistrial92• 3 years ago

fair, but include the "BlackSwan".. setbacks, destruction, etc

mmaunder2• 3 years ago

Got any historical examples? I lived through it from 1984 onwards as a young programmer and can't think of any, other than perhaps the dot-com bust (I was at eToys) which didn't impede the rate of technological progress much - only the rate of return.

arp2422• 3 years ago

I think history has shown us that we tend to underestimate the rate of technological progress and it's rate of acceleration.

It's also been overestimated tons of times. Look at some of the predictions from the past. It's been a complete crap-shoot. Many things have changed significantly less than people have predicted, or in significantly different ways, or significantly more.

Just because things are accelerating great pace right now doesn't really mean anything for the future. Look at the predictions people made during the "space age" 1950s and 60s. A well-known example would be 2001 (the film and novel). Yes, it's "just" some fiction, but it was also a serious attempt at predicting what the future would roughly look like, and Arthur C. Clarke wasn't some dumb yahoo either.

The year 2001 is more than 20 years in the past, and obviously we're nowhere near the world of 2001, for various reasons. Other examples include things like the Von Braun wheel, predictions from serious scientists that we'd have a moon colony by the 1990s, etc. etc. There were tons of predictions and almost none of them have come true.

They all assumed that the rate of progress would continue as it had, but it didn't, for technical, economical, and pragmatic reasons. What's the point of establishing an expensive moon colony when we've got a perfectly functional planet right here? Air is nice (in spite of what Spongebob says). Plants are nice. Water is nice. Non-cramped space to live in is nice. A magnetosphere to protect us from radiation is nice. We kind of need these things to survive and none are present on the moon.

Even when people are right they're wrong. See "Arthur C Clarke predicts the internet in 1964"[1]. He did accurately predict the internet; "a man could conduct his business just as well from Bali as London" pretty much predicts all the "digital nomads" in Bali today, right?

But he also predicts that the city will be obsolete and "seizes to make any sense". Clearly that part hasn't come true, and likely never will. Can't "remotely" get a haircut, or get a pint with friends, or all sorts of other things. And where are all those remote workers in Bali? In the Denpasar/Kuta/Canggu area. That is: a city.

It's half right and half wrong.

The take-away is that predicting the future is hard, and that anyone who claims to predicts the future with great certainty is a bullshitter, idiot, or both.

[1] https://www.youtube.com/watch?v=wC3E2qTCIY8

idopmstuff3• 3 years ago

What's the point of establishing an expensive moon colony when we've got a perfectly functional planet right here?

I think this is the big difference between what you're describing and AI. AI already exists, unlike a moon colony, so we're talking about pushing something forward vs. creating brand new things. It's also pretty well established that it's got tremendous economic value, which means that in our capitalist society, it's going to have a lot of resources directed at it. Not necessarily the case for a moon colony whose economic value is speculative and much longer term.

abeppu2• 3 years ago

It seems wild to me that the 50% prediction for "unaided machines outperforming humans in every possible task" is 2047, but for "all human occupations becoming fully automatable" is 2116. That's multiple generations of people going to work (or _training_ to go to work) knowing that a machine could do it better. Someone born after machines can outperform humans at _everything_ could go to school, work a series of jobs, and reach retirement age, if the median predictions are right. What would have to be true of human institutions or the economy for that to be true?

Thousands of AI Authors on the Future of AI

Related Stories