HN user

dncrane

195 karma
Posts6
Comments33
View on HN

But these heralds who have volunteered their clairvoyant services to us [...] just know that [superintelligence] is going to happen soon, if not already. And they do know, with all their hearts, that it is going to be bad. Really bad.

This is an unfair characterization of the the most prominent figures in this area. This article by Allan Dafoe and Stuart Russell from 2016 refutes it: https://www.technologyreview.com/s/602776/yes-we-are-worried...

For one, so far everyone predicting doom about AI has been a layman a subject.

This is a myth. It was arguably true 5-10 years ago, but concern with AI safety is not a fringe position even among the highest levels of AI researchers now.

Stuart Russel (https://en.wikipedia.org/wiki/Stuart_J._Russell) is the co-author of one of the most popular AI textbooks in the world and he has repeatedly said that he thinks the alignment problem is important and that AI presents an existential risk: https://www.technologyreview.com/s/602776/yes-we-are-worried... https://www.youtube.com/watch?v=WvmeTaFc_Qw

Marcus Hutter (https://en.wikipedia.org/wiki/Marcus_Hutter) is another respected AI researcher who, along with, Tom Everitt (http://www.tomeveritt.se/) (a researcher at DeepMind, one of the most advanced AI companies in the world), is also working on the alignment problem: http://www.tomeveritt.se/papers/alignment.pdf

You can read through list of grants granted by the Future of Life institute for AI Safety research, almost all of which are to researchers associated with respected universities, not laymen, here: https://futureoflife.org/ai-safety-research/

I posted an earlier version of this about 2 years ago. Since then I've added a bunch of music theory exercises and created a system for teachers to use the exercises in their classes by giving assignments, tracking scores, etc.

The new music theory exercises use vexflow (https://github.com/0xfe/vexflow ) for drawing the staff in addition to midi.js for the sound. The teacher-account backend uses Django.

Good idea! There's an advanced option that puts a "listen" button next to each interval on the sidebar in the intervals exercise, but this would be a good more general feature to include for whenever someone answers incorrectly in any exercise.

Thanks, that's a really good point. I agree that the patterns generated could be improved, but haven't come up with a good automated way to generate an unlimited amount of better ones. Please email me any thoughts you have about the best way to do this, I'm definitely interested in improving these (my email is on my profile page).

SymbolHound 11 years ago

Hi, I'm the developer of SymbolHound. You're right, SymbolHound's index is a little lacking, especially for new languages. This is because it's based almost entirely on a stackoverflow data dump (http://blog.stackoverflow.com/2009/06/stack-overflow-creativ...) from when I first made it (2011). Swift was released in 2014, so there are no results. I'm planning to update it when I have the time.

the link downloads immediately and there is no way to get the de-obfuscated link

If you go to your downloads list you can right click and do "copy link address" (at least on chrome and firefox)

Here's a good list of highly-praised books. Most of them look like they could make it past the censors: http://stackoverflow.com/questions/1711/what-is-the-single-m...

From those I personally recommend "Structure and Interpretation of Computer Programs" and "Gödel, Escher, Bach". SICP might not make it through, but GEB definitely would (although its direct relevance to day-to-day work in web development may be questionable, it should do a lot to spark an interest in computer science).

"Singularity related articles and worries about "good" AI building makes me a bit uneasy when the performance of machines on many important tasks, e.g. object recognition, is dismal. So we have quite a way to go."

Yudkowsky (2008), "AI as a Negative and Positive Factor in Global Risk" http://singinst.org/upload/artificial-intelligence-risk.pdf addresses this in section 7 (though it's really worth it to read the whole paper): "The first moral is that confusing the speed of AI research with the speed of a real AI once built is like confusing the speed of physics research with the speed of nuclear reactions. It mixes up the map with the territory."

===============

"in (2) how do we define the "upper limits of intelligence", when even defining intelligence is problematic."

Given the huge number of reproducible cognitive biases humans are known to exhibit (http://wiki.lesswrong.com/wiki/Bias), it seems very unlikely that humans are optimally intelligent (one way to define intelligence, from the Omohundro paper below: 'We define “intelligent systems” to be those which choose their own actions to achieve specified goals using limited resources' so "more intelligent" means better at achieving those goals using limited resources)

===============

"(3), I don't know what sort of "economic arguments" are meant but as we all know becoming "more intelligent" is not a simple hill climbing process, in fact it's not clear how to go about it."

Omohundro (2011), "Rationally Shaped Artificial Intelligence" Makes the case for #3: http://selfawaresystems.files.wordpress.com/2011/10/rational... (summary: by "more intelligent" we mean "more capable of achieving goals." So any AI which has goals will act to become more intelligent so that it can more effectively achieve those goals)

I guess the issue is how likely you think a hard takeoff/FOOM scenario is: http://wiki.lesswrong.com/wiki/Intelligence_explosion

"Sure a human level computer AI gets 'free' speed doubling every 18 months, but so does the intelligence that surrounds an augmented human. Just look at how much more intelligence we have available to us now as compared to 20 years ago."

A motivated human-level AI could get free speed doubling a lot more quickly than every 18 months -- it could acquire more resources that already exist, it could increase the speed of progress in hardware improvements, and, perhaps most importantly, it might be able to improve itself to achieve a qualitatively superhuman intelligence rather than just quantitatively. Sections 3: "Underestimating the power of intelligence" and 7: "Rates of intelligence increase" of the Yudkowsky paper I linked before, "AI as a Negative and Positive Factor in Global Risk", address this well. http://singinst.org/upload/artificial-intelligence-risk.pdf

I think it's a mistake to compare an AI superintelligence to anything on earth currently, like market forces. An AI superintelligence could probably improve itself a lot faster than any market could.

I don't think anybody from the SIAI is disagreeing with you here. I don't think the kind of prediction you're talking about is part of the intelligence explosion hypothesis. A superintelligence doesn't need literal omniscience, or even something that close to it, in order to be much, much, more effective than humans at achieving goals.

The intelligence explosion discussed in the original LessWrong post is a separate issue from exponential growth, only loosely related, although folks like Ray Kurzweil constantly conflate the two.

Please don't dismiss all ideas about AI, intelligence explosions, and superintelligence (e.g. http://singinst.org/upload/artificial-intelligence-risk.pdf ) just because some related concepts may be misguided.

(Not saying that you are being dismissive, just pointing out a distinction that you or others may not be aware of.)

"This all sounds reasonable, however they are held back by the idea that AI must look like a really smart human that is remotely controlling a robot/computer interface."

I think you misunderstood their position. The only assumption is that the AI must have some goal, and the point is that, if there's no term for humans in that goal, then we will likely be destroyed, not due to malice from any superintelligence, but indifference -- we are made out of resources that the AI could use to achieve its goals.

I think you're vastly underestimating how much better than us at achieving goals a superintelligence would be. A superintelligence would be a lot more powerful than any augmented billionaire, in the same way that a human is more powerful than the head of any wolf pack. The worry isn't about slightly augmented humans or slightly superhuman intelligence.

See these two papers for good arguments as to why human-level AI will likely result in an "intelligence explosion":

Yudkowsky (2008), "AI as a Negative and Positive Factor in Global Risk" http://singinst.org/upload/artificial-intelligence-risk.pdf Chalmers (2010), The Singularity: A Philosophical Analysis http://consc.net/papers/singularityjcs.pdf