Would it be evil to build a functional brain inside a computer?

http://io9.com/would-it-be-evil-to-build-a-functional-brain-inside-a-c-598064996

by robg • 13 years ago

40 47 13 years ago

read• 13 years ago

Nevermind evil; malice is too ambitious of a goal! Incompetence is more likely to dominate.

A question that's 100x more important is would you manage to even finish the project? Big projects like that suffer notoriously from mismanagement that the chances of them succeeding could be as low as inversely proportional to their funding. A startup with 1/100th the funding has more chances of getting something useful working.

Vicarious with $15M funding for example has better odds at getting to the root of what you need to build a functional brain that doesn't involve such a supercomputer.

http://vicarious.com

wellboy2• 13 years ago

It doesn't matter if "this" project will succeed. This development is unstoppable, because there will be hundreds of projects and mad scientists trying to create human consciousness in machines.

As soon as there is a tangible goal to attain where no one has ever been before, people tend to become unstoppable :)

pyre2• 13 years ago

I'm still waiting for my anti-gravity. ;-)

Aqueous2• 13 years ago

Depends on whether you believe in Strong AI or Weak AI, whether a simulation is consciousness or merely behaves like consciousness.

And then, because you can't decide between those two philosophical frameworks, you quickly realize that you can't possibly prove beyond a reasonable doubt that a computer is or isn't conscious. If we can't prove that it isn't conscious, are we obligated to assume that it is?

I'm not entirely sure we are. It's like asking if we can't prove that there is no God, are we obligated - ethically - to behave as if there is one? Most agnostics and atheists would say absolutely not.

Not being able to prove the negation of a fact is not the same as proving a fact.

civilian• 13 years ago

"whether a simulation is consciousness or merely behaves like consciousness." Crucial point. Have you read Peter Watt's Blindsight? I can't recommend it enough. http://www.rifters.com/real/Blindsight.htm (it's also available in a pay-money format)

There's also the whole idea of whether humans are consciousness or merely behave like consciousness. There's the idea that the sentient "you" is just a narrator for the decisions that your brain makes. This is one example of that: http://www.nature.com/news/2008/080411/full/news.2008.751.ht...

Aqueous2• 13 years ago

'There's also the whole idea of whether humans are consciousness or merely behave like consciousness.'

Well this would be the counterargument. We behave ethically towards humans. Yet we have no conclusive evidence that other human beings are conscious. So is that a good enough reason to treat conscious-behaving computers ethically?

Thanks for the links.

wellboy2• 13 years ago

My take on this. As long as mankind has existed that things are the way they are, because of some magical thing and thousands of times, over and over again, it just wasn't magical, it could be explained through experiments and logical conclusion.

The same will happen for human consciousness. Does it sound more plausible that the human mind is conscious, because of billions of brain cells with trillions of connections or because of some magical thing?

The human mind is just an incredibly sophisticated machine and as soon as we can rebuild that, we will have an equally conscious computer.

dclowd9901• 13 years ago

If a consciousness can mimic fear, panic and anxiety about its own existence, that's all I need to put some protective rules in place. It has to grow organically out of the code. Even if it's all "simulated," it's still represented. Otherwise, you might as well assume all humans are faking their external emotions, which would then make you a sociopath.

One of the central tenets of our reality as human beings is that we can't truly know what it is to be another. We make up for it with empathy, and we should extend that to any being that we have the capacity of empathizing with through their shared external emotives.

cynoclast• 13 years ago

There's no such thing as evil.

More garbage from io9.com....

6d0debc0713• 13 years ago

While I think these sorts of bills of rights are important - after all, there may well come a point when things that are faster thinking than us are dominant, or when our descendants/ourselves are simulated in some form or another. To an extent, whether it is actually conscious or just simulates consciousness is irrelevant (and, IMO, likely to go unanswered anyway since you can hardly ask to see someone else's qualia.) If it behaves well if we treat it well - if it seems to develop ethics the same way we do - then there's a pretty darned good reason to treat it well. Golden rule, after all; do as you would be done by.

Semantics aside - if you could simulate a brain inside a computer without the environment, which seems the most likely first step, would you essentially be locking it in sensory deprivation? That seems like the sort of thing we shouldn't do.

kailuowang2• 13 years ago

It's interesting that people are trying to determine the ethics of simulating human brain while ethics itself is a probably a product of human brain.

crusso4• 13 years ago

In the same way that math is.

kailuowang2• 13 years ago

Can you elaborate on "the same way"? I thought most people would agree that math is not a product of human brain, it exists regardless of the existence of human race.

sliverstorm2• 13 years ago

"Math" is a human invention much the same way "Time" is a human invention. We measure it, we describe it, we manipulate it on paper- but It was always there.

coopdog• 13 years ago

I think the bigger question is when do we grant an information automata the right to survive? If we use evolutionary algorithms with survival as a selection bias (just like humans) eventually creatures that 'think' like us should emerge.

Whereas there are intelligent automata, like say a search engine, that absolutely could not care either way whether you turn it off or not. The survival selection bias is the difference, and it's seeing something struggle to survive that activates our empathy emotions to save it.

I imagine we'll know the line of when it would be 'evil' to go against our own instincts and turn off an intelligent automata that wants to survive when we see it

daniel-cussen• 13 years ago

I've erased millions, if not billions, of information automata while working with evolutionary algorithms. How do I know I'm not killing something living, when the whole idea is that they truly live?

I don't. While in retrospect they weren't complex enough to really think, I'm so sure that they'll be unfriendly AI that I made the decision beforehand to erase them no matter what, "dead or alive."

brucefancher• 13 years ago

You monster.

VladRussian2• 13 years ago

I imagine we'll know the line of when it would be 'evil' to go against our own instincts and turn off an intelligent automata that wants to survive when we see it

considering how we treat other intelligent lifeforms around us on the mere basis that we perceive their intelligence inferior to us, i'd be surprised if we find "the line" for the intelligent automata. Well, i guess until it forces it upon us.

jimrandomh• 13 years ago

If someone eventually does manage to build a functional human brain inside a computer, that first brain will raise much bigger issues. You see, that person in the computer, whoever they are, will have automatically acquired certain powers - including, in particular, self-copying, self-modification, and (with enough processing power), accelerated thought.

In other words, they'll be a motherfucking sorcerer. One who's just been through an untested brain-to-software translation, with possible sanity damage. Now, keeping in mind that we're talking about a motherfucking sorcerer with possible sanity damage, reread that list:

    1. The right to not be shut down against one’s will
    2. The right to not be experimented upon
    3. The right to have full and unhindered access to one’s own source code
    4. The right to not have one’s source code manipulated against their will
    5. The right to copy (or not copy) oneself
    6. The right to privacy (namely the right to conceal one’s own internal mental states)
    7. The right of self-determination

This is an extraordinarily bad idea. If implemented, it would almost certainly wipe out humanity.

aperrien2• 13 years ago

I believe we need to take steps now to better get a handle on what we define as "sane".

If this is a person who has been placed in a computer, it's unlikely that they'll be willing to wipe out all their friends and family on a whim.

pyre2• 13 years ago

How many of your feelings are responses to chemical reactions? How well will these be emulated? Will it just be a sort of complex 'neural net' (mapping of neurons) or will all of the chemical processes be there?

jared314• 13 years ago

That depends on what survives the transfer (i.e. memories, experiences, and/or trained personal responses). I think we can actually use existing sci-fi to explore the possibilities, ranging from nothing to everything surviving. Even Portal has a perspective on building an AI from, or including, a person[0].

[0] http://theportalwiki.com/wiki/Caroline

fleitz• 13 years ago

Who is to say humanity is the final product of evolution in the universe? Most people don't consider humanity to be evil for wiping out neanderthality.

freehunter• 13 years ago

Yeah but we're not facing down imminent extinction as Neanderthals. In this hypothetical situation, we are facing imminent extinction as humans. When it's happening to you it's kind of a big deal.

marcosdumay• 13 years ago

Who is to say that humans should help any other species to become the next step of our evolution?

6d0debc0713• 13 years ago

We'd probably consider N pretty stupid if it just rolled over and died though.

cr4zy• 13 years ago

Copying a brain from a scan will probably take much longer than efforts to create intelligence from first principles (think of making a flying machine via reconstructing feathers, wings, etc...) However, in order to create a functional human-like brain inside a computer, we will most likely need to train it like a human brain using the same principles with use to teach humans. In so doing, we can endow it with the same morals and ethics we do our own children. Also, since early prototypes will be less-powerful than ours, we'll be able to test the morality of such intelligence in a much more gradual way.

Btw, I'm creating a brain prototype if you're interested in that sort of thing. Here's where most of the structure lies (https://github.com/crizCraig/brain/blob/master/src/neuron.py)

6d0debc0713• 13 years ago

Possible sanity damage - well, if they've got it wrong I'd expect them not to have a functional brain simulation at all. But maybe.

I wouldn't be so sure that they'd immediately be all that powerful though. Just because you're running on a certain architecture doesn't mean you understand the first thing about it or how to modify it - just think how long it's taking humans to understand and alter neurology.

Admittedly they might have a speed advantage, but who knows what they'll use that for - depending on the person and the environment they might spend the next thousand years taking e-heroin, or playing tick tack toe.

And, if they really are an insane sorcerer, do you want them to think that you're not going to respect their rights?...

Though, this does strike me as a good argument for only running the initial upload on crippled systems - no access to the internet, limited speed, etc.

galapago• 13 years ago

No.

unclebucknasty• 13 years ago

Have we even answered the question, "what is consciousness"? Isn't this along the lines of "what is the meaning of life"?

Because, if we haven't answered it, then it seems that the best we could do is build a simulation which displays some or many of the properties that we associate with consciousness.

hobs• 13 years ago

Yeah, two things on that: 1. I know we are making leaps and bounds with computers, but we simply do not understand ourselves enough to make a copy of it. If you say that we can just simulate the physical capabilities to the atomic level, good luck to you. 2. So many questions about an electronic brain and yet it will (for a long time) have plenty less brain cells than a cow or a cuttlefish, and yet we kill and eat both every day(Delicious meat, I must eat).

So, when does the cow end and a human computer program begin? Do we need humane programming practices?

jes51992• 13 years ago

No, but, there are people coming up with theories on how to quantify and measure it anyway. For example, http://en.wikipedia.org/wiki/Integrated_Information_Theory

aperrien2• 13 years ago

The ethics of whether or not to allow independent, non-human originating agents to be built is highly debatable, for obvious reasons.

However, as to whether or not to run an emulation of someone who has volunteered their brain to such an experiment is much less murky. As long as the experiments have been agreed upon in advance, and carried out with a neutral third party observing and acting as a control, I don't see too many problems. We do that sort of experimenting with humans as a matter of course, and people being biologically based, or electronically based should not matter.

contingencies• 13 years ago

Run for the hills.

spydum• 13 years ago

Can't wait for the AI to file a lawsuit against the developer for defects found in his/her/it's? implementation

jes51992• 13 years ago

Can you sue your parents for faulty genetics?

flashmedium• 13 years ago

A) No B) It's unstoppable. For as long as humans will have the desire to live a longer and healthier life... Piles of money will be poured into R&D.

Bionic limbs, eyes... down the line bionic mind. Human, meet Cyborg.

pbw2• 13 years ago

It will entirely be a question of politics, not science or philosophy. The track record is we deny rights to those we deem too different from us, then we later see the light and include them. Think about women or other races. This will happen with virtual beings. There will never be any proof they are conscious, there can't be, but we'll have to decide anyway how to label them. We won't agree that is for sure, it will be like the debate around abortion only more divisive and more significant. We will never definitively decide, we'll still be debating right on into our obsolescence.

guscost• 13 years ago

Would you agree that today many people ascribe consciousness to dogs and other mammals? Keeping in mind that most mammals don't speak English, I'd assume that many people would befriend this hypothetical technology in the "dog phase" or earlier.

Also, as a group women and other races have been considered more different, then less different, without actually changing from one to the other. The same couldn't necessarily be said about AI projects.

pbw2• 13 years ago

Most people believe their dogs are less conscious than they are, but more conscious than gold fish, etc. I agree AI will achieve various levels of consciousness at various times, starting well below us and ending up well above us.

DougN7• 13 years ago

Am I the only one that thinks this is silly? A computer is still just a Turing machine. It's a gigantic abacus, or a huge clock of gears. Heck, I can write a program today to output text begging the power not to be shut off. That certainly doesn't mean it's self-aware.

Would it be evil to build a functional brain inside a computer?

Related Stories