Analyzing a Critique of the AI 2027 Timeline Forecasts

https://thezvi.substack.com/p/analyzing-a-critique-of-the-ai-2027

by jsnider3 • 1 year ago

38 61 1 year ago

thezvi.substack.com

f38zf5vdt• 1 year ago

I think the author is right about AI only accelerating to the next frontier when AI takes over AI research. If the timelines are correct and that happens in the next few years, the widely desired job of AI researcher may not even exist by then -- it'll all be a machine-based research feedback loop where humans only hinder the process.

Every other intellectual job will presumably be gone by then too. Maybe AI will be the second great equalizer, after death.

goatlover2• 1 year ago

Except we have no evidence of AI being able to take over AI research anymore than we have evidence so far that automation this time will significantly reduce human labor. It's all speculation based on extrapolating what some researchers think will happen as models scale up, or what funders hope will happen as they pour more billions into the hype machine.

dinfinity5• 1 year ago

It's also extrapolating on what already exists. We are way beyond 'just some academic theories'.

One can argue all day about timelines, but AI has progressed from being fully inexistent to a level rivaling and surpassing quite some humans in quite some things in less than 100 years. Arguably, all the evidence we have points to AI being able to take over AI research at some point in the near future.

KaiserPro2• 1 year ago

bangs head against the table.

Look, fitting a single metric to a curve and projecting from that only gets you a "model" that conforms to your curve fitting.

"proper" AI, where it starts to remove 10-15% of jobs will cause an economic blood bath.

The current rate of AI expansion requires almost exponential amounts of cash injections. That cash comes from petro-dollars and advertising sales. (and the ability of investment banks to print money based on those investment) Those sources of cash require a functioning world economy.

given that the US economy is three fox news headlines away from collapse[1] exponential money supply looks a bit dicey

If you, in the space of 2 years remove 10-15% of all jobs, you will spark revolutions. This will cause loands to be called in, banks to fail and the dollar, presently run obvious dipshits, to evaporate.

This will stop investment in AI, which means no exponential growth.

Sure you can talk about universal credit, but unless something radical changes, the people who run our economies will not consent to giving away cash to the plebs.

AI 2027 is unmitigated bullshit, but with graphs, so people think there is a science to it.

[1] trump needs a "good" economy. If the fed, who are currently mostly independent need to raise interest rates, and fox news doesn't like it, then trump will remove it's independence. This will really raise the chance of the dollar being dumped for something else (and its either the euro or renminbi, but more likely the latter)

That'll also kill the UK because for some reason we hold ~1.2 times our GDP in US short term bonds.

TLDR: you need an exponential supply of cash for AI 2027 to even be close to working.

goatlover2• 1 year ago

It's certainly hard to imagine the political situation in the US resulting in UBI anytime soon, while at the same time the party in control wants unregulated AI development for the next decade.

bcrosby95• 1 year ago

It's the '30s with no FDR in sight. It won't end well for anyone.

gensym• 1 year ago

AI 2027 is unmitigated bullshit, but with graphs, so people think there is a science to it.

AI 2027 is classic Rationalist/LessWrong/AI Doomer Motte-Bailey - it's a science fiction story that pretends to be rigorous and predictive but in such a way that when you point out it's neither, the authors can fall back to "it's just a story".

At first I was surprised at how much traction this thing got, but this is the type of argument that community has been refining for decades and this point, and it's pretty effective on people who lack the antibodies for it.

tux3• 1 year ago

It's the other way around entirely: the story is the unrigorous bailey, when confronted they fall back to the actual research behind it

And you can certainly criticize the research, but you've got the motte and the bailey backwards

mitthrowaway23• 1 year ago

I'm very much an AI doomer myself, and even I don't think AI 2027 holds water. I find myself quite confused about what its proponents (including Scott Alexander) are even expecting to get from the project, because it seems to me like the median result will be a big loss of AI-doomer credibilty in 2028 when the talking point shifts to "but it's a long tailed prediction!"

stego-tech3• 1 year ago

It got traction because it supported everyone’s position in some way:

* Pro-safety folks could point at it and say this is why AI development should slow down or stop

* LLM-doomer folks (disclaimer: it me) can point at it and mock its pie-in-the-sky charts and milestones, as well as its handwashing of any actual issues LLMs have at present, or even just mock the persistent BS nonsense of “AI will eliminate jobs but the economy [built atop consumer spending] will grow exponentially forever so it’ll be fine” that’s so often spewed like sewage

* AI boosters and accelerationists can point to it as why we should speed ahead even faster, because you see, everyone will likely be fine in the end and you can totes trust us to slow down and behave safely at the right moment, swearsies

Good fiction always tickles the brain across multiple positions and knowledge domains, and AI 2027 was no different. It’s a parable warning about the extreme dangers of AI, but fails to mention how immediate they are (such as already being deployed to Kamikaze drones) and ultimately wraps it all up as akin to a coin toss between an American or Chinese Empire. It makes a lot of assumptions to sell its particular narrative, to serve its own agenda.

OgsyedIE• 1 year ago

I disagree with the forecast too, but your critique is off-base. The assumption that exponential cash is required assumes that subexponential capex can't chug along gradually without the industry collapsing into mass bankruptcy. Additionally, the investment cash that the likes of Softbank are throwing away comes from private holdings like pensions and has little to nothing to do with the sovereign holdings of OPEC+ nations. The reason that it doesn't hold water are the bottlenecks on compute production. TSMC is still the only supplier of anything useful for foundation model training and their expansions only appear big and/or fast if you read the likes of Forbes.

KaiserPro2• 1 year ago

cash that the likes of Softbank are throwing away comes from private holding

Softbank use debt to leverage various things. They require a functioning monetary system to work. They cannot work as they are in a financial recession. Sure some of it might pensions, but again that drys up when the financial system freezes.

Money doesn't exist in a vacuum. Its not some fixed supply of things. its a dynamic system that grows and contracts based largely on vibes.

pier254• 1 year ago

AI 2027 is unmitigated bullshit, but with graphs, so people think there is a science to it.

One of the best things I've read all day.

JimDabell• 1 year ago

It’s not just changing economics that will derail the projections. The story gives them enough compute and intelligence to massively sway public opinion and elections, but then seems to just assume the world will just keep working the same way on those fronts. They think ASI will be invented, but 60% of the public will disapprove; I guess a successful PR campaign is too difficult for the “country of geniuses in a datacenter”?

jvalencia• 1 year ago

It's like the invention of the washing machine. People didn't stop doing chores, they just do it more efficiently.

Coders won't stop being, they'll just do more, compete at higher levels. The losers are the ones who won't/can't adapt.

falcor84• 1 year ago

I suppose that those who stayed in the washing business and competed at a higher level are the ones running their own laundromats; are they the big winners of this technological shift?

alganet2• 1 year ago

What are you even talking about?

The article is not about AI replacing jobs. It doesn't even touch this subject.

fasthands9• 1 year ago

Yeah. For understandable reasons that is covered a lot too, but AI 2027 is really about the risk of self-replicating AI. Is an AI virus possible, and could it be easily stopped by humans and our military?

bgwalter• 1 year ago

No, all washing machines were centralized in the OpenWash company. In order to do your laundry, you needed a subscription and had to send your clothes to San Francisco and back.

jgalt212• 1 year ago

Excellent analogy

vntok• 1 year ago

Exactly, it wasn't the case then with washing machines and it's not the case now with AI. Your example is pretty relevant!

Today, anyone can run SOTA open-weights models in the comfort of their home for much less than the price of a ~1929 electric washing machine ($150 then or $2,800 today).

stego-tech3• 1 year ago

Reading through the comments, I am so glad I’m not the only one beyond done with these stupid clapbacks between boosters and doomers over a work of fiction that conveniently ignores present harms and tangible reality in knowledge domains outside of AI - like physics, biology, economics, etc.

If I didn’t know better, it’s almost like there’s a vested interest in propping these things up rather than letting them stand freely and let the “invisible hand of the free market” decide if they’re of value.

old_man_cato4• 1 year ago

Sometimes I feel like I'm losing my mind with this shit.

Am I to understand that a bunch of "experts" created a model, they surrounded the findings of that model with a fancy website, replete with charts and diagrams, that website suggests the possibility of some doomsday scenario, the headline of the website says "We predict that the impact of superhuman AI over the next decade will be enormous, exceeding that of the Industrial Revolution." WILL be enormous. Not MIGHT be, they went on some of the biggest podcasts in the world talking about it, a physicist comes along and says yeah this is shoddy work, the clap back is "Well yeah it's an informed guess, not physics or anything"?

What was the point of the website if this is just some guess? What was the point of the press tour? I mean are these people literally fucking insane?

refulgentis4• 1 year ago

Correct. Entirely.

And I'm yuge on LLMs.

It is very much one of those things that makes me feel old and/or scared, because I don't believe this would have been swallowed as easily, say, 10 years ago.

As neutrally as possible, I think everyone can agree:

- There was a good but very long overview of LLMs from an ex-OpenAI employee. Good stuff, really well-written,

- Rapidly it concludes by hastily drawing a graph of "relative education level of AI" versus "year", draw a line from high school 2023 => college grad 2024 => phd 2025 => post-phd 2026 => agi 2027.

- Later, this gets published by same OpenAI guy, then the SlateStarCodex guy, and some other guy.

- You could describe it as taking the original, cut out all the boring leadup, jumped right to "AGI 2027", then wrote out a too-cute-by-half, way too long, geopolitics ramble about China vs. US.

It's mildly funny to me, in that yesteryear's contrarians are today's MSM, and yet, they face ~0 concerted criticism.

In the last comment thread on this article, someone jumped in to discuss the importance of more "experts in the field" contributing, meaning, psychiatrist Scott Siskind. The idea is writing about something makes you an expert, which leads us to tedious self-fellating like Scott's recent article letting us know LLMs don't have to have an assistant character, and how he predicted this years ago

It's not so funny, in that the next time a science research article is posted here, as is tradition, 30% will be claiming science writers never understand anything and can't write etc. etc.

heavyset_go3• 1 year ago

The point? MIRI and friends want more donations.

old_man_cato4• 1 year ago

Well, yeah. Obviously.

shaldengeki2• 1 year ago

No, you're wrong. They wrote the story before coming up with the model!

In fact the model and technical work has basically nothing to do with the short story, aka the part that everyone read. This is pointed out in the critique, where titotal notes that a graph widely disseminated by the authors appears to be generated by a completely different and unpublished model.

old_man_cato4• 1 year ago

https://ai-2027.com/research says that:

AI 2027 relies on several key forecasts that couldn't be fully justified in the main text. Below we present the detailed research supporting these predictions.

You're saying the story was written, then the models were created and the two have nothing to do with one another? Then why does the research section say "Below we present the detailed research supporting these predictions"?

radioactivist• 1 year ago

Thank you for this comment, it is exactly my impression of all of this as well.

shitloadofbooks• 1 year ago

AI proponents keep drawing perfectly straight lines from "no AI --> LLMs exist --> LLMs write some adequate code sometimes" up into the horizon of the Y axis where AIs run all governments, write all code, paint all paintings and so on.

There's a large overlap with the crypto true-believers who were convinced after seeing "no blockchain --> blockchain exists" that all laws would be enshrined in the blockchain, all business would be done with blockchains, etc.

We've had automation in the past; it didn't decimate the labour-force; it just changed how people work.

And we didn't go from handwashing clothes --> washing machines --> all flat surfaces are cleaned daily by washing robots...

refulgentis4• 1 year ago

Would advise, generally, that AI isn't crypto.

It's easy to lapse into personifying it and caricaturing the-thing-in-toto, but then we end up at obvious absurdities - to wit:

- we're on HN, it'd be news to most readers that there's a "large overlap" of "true-believers", AI was a regular discussion topic here a loooong time before ChatGPT, even OpenAI. (been here since 2009)

- Similarly "AI proponents keep drawing perfectly straight lines...AIs run all governments, write all code, paint all paintings and so on."

The technical term would be "strawmen", I believe.

Or maybe begging the question (who are these true-believers who overlap? who are these AI proponents)

Either way, you're not likely to find these easy-to-knock-down caricatures on HN. Maybe some college hypebeast on Twitter. But not here.

mystified5016• 1 year ago

I have personally seen all of these people on HN.

gmuslera• 1 year ago

My main objection against this kind of predictions is that predictions (at least, the known enough ones) are part of the past that shape the future. Even doing a good extrapolation on the current trends, the prediction itself could make things diverge, converge or do something totally different, because the main decision makers will take it into account and that is not part of the trend. Specially with disruptive enough predictions that paints an undesirable future for all or most decision makers.

Unless it hits hard in some of the areas that we have cognitive biases and are not fully rational on the consequences.

Analyzing a Critique of the AI 2027 Timeline Forecasts

Related Stories