The Myth of a Superhuman AI 5 years ago
This is a pretty bad article.
Agreed.
Dunno why he thinks this: "At the core of the notion of a superhuman intelligence — particularly the view that this intelligence will keep improving itself — is the essential belief that intelligence has an infinite scale."
The notion of self-improvement is both powerful and not assumed to be 'infinite'.
I figure it goes like this:
Q. Is AI potentially dangerous? A. Yes.
Q. Will it be human-like? A. Probably not.
People tend to couch these discussions as a series of anthropomorphisms.