George Eliot's reflections on AGI from 1879

https://www.learningfromexamples.com/p/reflections-on-superintelligence

by benbreen • 1 year ago

145 56 1 year ago

pama• 1 year ago

With the explosion of LLMs we are close to the final thoughts in her chapter: “this planet may be filled with beings who will be blind and deaf as the inmost rock, yet will execute changes as delicate and complicated as those of human language and all the intricate web of what we call its effects, without sensitive impression, without sensitive impulse.”

XorNot• 1 year ago

The thing which has become very weird for me is watching Star Trek holodeck episodes in the age of LLMs. When I was in high school we considered this absurd writing - obviously they must be sentient...now LLMs have happened and it feels obvious that the opposite can be true.

Same story with Star Wars and droids: C3PO wasn't sentient, just a protocol model loaded into a droid frame in the wrong environment, it's attempts to constantly being the conversation back to politeness being a simplistically coded filter layer.

optimalsolver• 1 year ago

Another thing Trek (and other SF) didn't predict was that AI would speak with Bay Area vocal fry.

Someone should re-dub the Enterprise computer's lines with it.

captainclam3• 1 year ago

Haha, I've had the same thoughts, that of course computers/AI/droids of that conversational capacity were conscious. You'd be a brute not to think that!

And all of a sudden, LLMs absolutely have the command of natural language that once seemed such an obvious indicator of sentience, and now I find myself one of those bigots who don't believe in robot rights!

I'm being silly, but I do think there are implications here with respect to the future debate on AI sentience. I guess I once thought there would be this threshold where the reality of an AI's inner experience became blatantly obvious, but I see now that this is going to be a profoundly thorny problem.

Who knows, maybe in several decades we'll have a consciousness-o-meter that demonstrates that LLMs have had some degree of awareness all along.

jameshart• 1 year ago

Threepio reads so much like an LLM, the writing turns out to have been skockingly prescient. At the time of Phantom Menace, the whole story of him being assembled by some slave kid on a backwater planet in the outer rim seemed preposterous - why would a slave kid into pod racing make a pompous ass of a protocol droid? Well, he just installed a standard easily pirated LLM into the droid he made.

And the fact that these LLMs are used as 'protocol droids' to communicate with lower-level devices - 'the binary language of moisture vaporators' etc. - matches up with the pattern of LLMs as tools for generating scripts in Python.

Terrible though the last sequel is, they even managed to include jailbreaking Threepio to get him to access part of his training data that had been censored out.

nthingtohide3• 1 year ago

Philosophers have coined the term P-zombies. LLMs can be considered as P-zombies if they do not possess consciousness. Functionally equivalent but no soul. Substance Dualism was used by philosophers to deny animals consciousness.

farts_mckensy• 1 year ago

Non materialist bullshit.

kbelder• 1 year ago

Functionally equivalent but no soul.

I think it's an incoherent concept. If the 'soul' is anything, there is some difference in the brain that would allow it. You can't have two people with identical brains and claim that one has a soul, unless you believe in magic.

_wire_• 1 year ago

Re dualism:

To paraphrase Chonsky (link below), the legacy of Descartes' observations for "mechanical philosophy" is profound: substance-dualism became untenable because no coherent idea of material can be formulated. Newton exorcised the machine (material), leaving the ghost intact to be curious about itself.

Today we study the organism called the body without need of a philosophical grounding in any reality. "mechanical philosophy" ended with Newton's work obviating any coherent notion of an "intelligible world". Faced with unreconcilable confusion about what constitutes reality, science quietly abandoned the idea and moved ahead with "intelligible theories about the world", which Chomsky regards as one of most significant shifts in science.

https://m.youtube.com/watch?v=bWDJ2zFe4Pc

In various speeches and lectures on the history of science and linguistics, Chomsky comments on artificial intelligence, by referring to Alan Turing's seminal paper, in which Turing proposed the so-called "test." Chomsky refers to the paper's first paragraph where Turing concedes that the meaning of the word "intelligence" is so indistinct as to be left to be determined by Gallop. Turing punts on the entire question of the meaning of intelligence by suggesting an "imitation game."

Today's state of the art of machine intelligence has not advanced past Turing's proposal. Chomsky wonders if we may have to resign ourselves to the limit that it never will, in which light of reason we might embark on further study of ourselves, rather than contrive facsimiles and endure all the attendant hazards.

unwise-exe2• 1 year ago

> Functionally equivalent but no soul.

Something that predicts no observable differences is unfalsifiable aka bullshit.

superkuh2• 1 year ago

Basically the plot of Peter Watts' "Blindsight" (post-2000) scifi book series.

antonkar2• 1 year ago

Before creating “eGod”, it’s safer to first create “eHeaven”. I propose building direct-democratic multiversal static place intelligence (you can google it): the static place AI, the place of eventual all-knowing, where we are the only agents that will eventually become all-powerful. We start with making a digital copy of Earth and some ideally wireless brain-computer-interfaces, everyone who wants can buy a special armchair for their living room and go to the simulated planet with the whole experience the same as on our planet, but when a bus hits you there, you just open your eyes. So it gives you immortality from injuries. And of course there will be may other worlds with different freedoms, like flying or doing magic or instant teleportation in public places…

fishtacos2• 1 year ago

Sort of related, but in the two or three times I've lucid dreamt, one of them was flying and having relatively functional control of my direction. Not sure how/why my brain rendered this sensation having never experienced it in real life, but it felt strange and novel and not easy to direct.

It was akin to a sensation of VR games where my mind seems to interpolate real-life expectations with visual input, but not quite. Not quite "brain-computer-interfaces", but perhaps a glimpse with current tech.

antonkar2• 1 year ago

Yes, that's great, I jumped over building in my dreams a few times and it was me controlling it, I think it'll resemble some future "multiversal UI" a bit. Did it look anything like long-exposure photos? https://wesely.org/2019/flughafen-tempelhof-berlin-1-7-2008-... There are more good photos like this in the end of this article: https://www.lesswrong.com/posts/LaruPAWaZk9KpC25A/rational-u...

sho_hn• 1 year ago

This was a joy to read.

cess11• 1 year ago

https://www.gutenberg.org/cache/epub/10762/pg10762-images.ht...

It comes across as satire to me, which I did not expect after reading TFA.

From the classical references littered throughout the portions I've skimmed, it seems like a learned person is joking about contemporary men that aren't as well educated and unaware of it.

Quite reminiscent of some of the stuff Kierkegaard wrote.

joshmarlow2• 1 year ago

Interesting - I wonder if the author was gesturing at Erewhon which was published a few years earlier. https://en.wikipedia.org/wiki/Erewhon

alabastervlog• 1 year ago

For those considering them, Erewhon and Erewhon Revisited (wherein we discover just how much in the way of, ahem, relations a book of the period can pack "between the lines") are available in a single widely available and cheap (used) little Modern Library hardback, and are a pretty fun read for the genre (nothing like Utopia—I read a lot of things most folks would consider boring, and that one taxed even my patience).

stevenwoo• 1 year ago

I recently reread Middlemarch, and this time I had finally read most of the literature and had wikipedia on hand to learn about the historical events that are referenced in it, the breadth and depth of her erudition is impressive, as is her portrayal of human nature (complicated and believable communicated across the divide of culture and time), which is the main thing I took from my first reading.

joshmarlow2• 1 year ago

Fascinating read!

Theophrastus closes by asking whether human consciousness is a ‘stumbling’ on the way to ‘unconscious perfection’.

This reminds me of a line from the scifi novel Echopraxia where one of the characters explains that as AIs become more intelligent, they 'wake up' and become conscious like humans. But then as they continue to grow, they eventually go back to sleep - and those are the machines to be afraid of. '

fishtacos2• 1 year ago

I'm not well-versed enough on sci-fi to be able to connect more dots than this, but I am assuming these are common tropes.

The Mass Effect series describes the Reapers as (copied from masseffect.fandom.com - I <3 this game's lore):

"The Reapers are a highly-advanced machine race of synthetic-organic starships. The Reapers reside in dark space: the vast, mostly starless space between galaxies. They hibernate there, dormant for fifty thousand years at a time, before returning to the galaxy...the Reapers spare little concern for whatever labels other races choose to call them, and merely claim that they have neither beginning nor end."

The other pop-sci-fi analogue I can think of is The Borg.

le-mark• 1 year ago

Fred Saberhagen Beserkers are the first instance of this concept I’m familiar with (1963)

https://en.m.wikipedia.org/wiki/Berserker_(novel_series)

goatlover3• 1 year ago

In the 3001 novel, the last of the 2001 series by Clarke, the Dave simulation explains that while the monolith can simulate conscious beings, it is itself not conscious. This is similar to The Expanse novels, where it's explained that that the protomolecule technology was limited to not being conscious on purpose by it's creators, and yet could still somehow simulate conscious beings it wanted to use as tools.

gwern• 1 year ago

Not many people have Eliot down as an influential player in the history of AI — and they would be right. Her contribution has been for most part overlooked, even as folks begin to unpick the role of fiction shaping the trajectory of the AI industry.

It's because it reads mostly as a retread of the far-better known Samuel Butler pieces like _Erewhon_ years before (https://en.wikipedia.org/wiki/Darwin_among_the_Machines was 1863), and which Eliot likely read given her lifestyle & how closely she echoes him. Surprising to see no mention of Butler at all in OP, given what critical context it is - she was not writing in a vacuum.

Some earlier discussion: https://www.lesswrong.com/posts/goANJNFBZrgE9PFNk/shadows-of...

setgree• 1 year ago

I learned about Butler and his letter from George Dyson's book, "Darwin among the Machines: The Evolution of Global Intelligence" -- it's a very entertaining account. Butler was quite a character.

(If I recall correctly I learned about Dyson's book from a postscript to Neal Stephenson's "Quicksilver.")

MichaelZuo2• 1 year ago

It does seem a bit suspicious given the context, as if the author intentionally omitted the background.

hiatus• 1 year ago

"Good artists copy, great artists steal." -- hiatus

twic• 1 year ago

I wonder if the title "The Shadows of the Coming Race" alludes to "Vril: The Power of the Coming Race", published eight years before, if that phrase was already in circulation. Google ngrams first sees it in 1868.

anon84873628• 1 year ago

Sure seems likely! Here's an excerpt from the Wikipedia article for Vril:

When H. G. Wells' novella The Time Machine was published in 1895, The Guardian wrote in its review: "The influence of the author of The Coming Race is still powerful, and no year passes without the appearance of stories which describe the manners and customs of peoples in imaginary worlds, sometimes in the stars above, sometimes in the heart of unknown continents in Australia or at the Pole, and sometimes below the waters under the earth. The latest effort in this class of fiction is The Time Machine, by H. G. Wells."

So the phrase "the Coming Race" must have been well established in literary circles, and probably used as a shorthand for what today we might call non-humans or post-humans.

timewizard• 1 year ago

any physical or cognitive task there aren’t exactly many roles that a human being can play that are additive to the economy.

What a terribly small imagination this presents. There is life outside of your cubicle. There always was.

Each time a person engages in work that could be done more efficiently by a machine, the employing party—be that company, government or charity—will have to bear the opportunity cost of using AI.

Your LLMs run on GPUs and have no physical component. Those physical components aren't free, require maintenance, and break at inconvenient times. This is not a zero sum future or one that doesn't require intelligent economic planning.

It’s like when a toddler ‘helps’ you wash the dishes.

It's cute you presume your "AGI" is going to /want/ to do anything for you. In particular, mundane chores, like washing your dishes, or that you as an individual would be able to afford it.

Anyways, it's an interesting past time around here, rather than admit the gulf between the current technologies and what AGI is imagined to be, we just troll through the past looking for anyone who even remotely made a "thinking machine" part of a fictional story.

Well, one part she got right, the machine presents itself as something of a useless chat bot.

unwise-exe2• 1 year ago

If you want to know what life is like when society is run by beings smarter than you, ask someone who flunked out of high school.

If you want to know what life is like when society is run by alien intelligences, ask someone who lives under an economy dominated by megacorps.

katzenversteher• 1 year ago

Damn, it's pretty early while I'm reading this and in my tired state, not noticing the year in the title, I somehow expected an article about Sierra adventure game internals... A bit disappointed but still a good read on a completely different topic :)

superkuh2• 1 year ago

Another early book on the implications of AI is Samuel Butler's "Erewhon" (1872) satire which describes a society in which a complex machine uprising occurred and thereafter all complex mechanical/computational devices were banned in their society. This idea is later referenced in the well known 20th century "Dune" books as "The Butlerian Jihad".

George Eliot's reflections on AGI from 1879

Related Stories