AlphaGo shows its true strength in 3rd victory against Lee Sedol

https://gogameguru.com/alphago-shows-true-strength-3rd-victory-lee-sedol/

by luu • 10 years ago

271 222 10 years ago

gogameguru.com

greenyoda• 10 years ago

The main discussion of the third game can be found here: https://news.ycombinator.com/item?id=11271816

However, this article contains much more analysis of the game than previously posted articles, and is worth reading.

atrudeau2• 10 years ago

I felt that Lee thought global sometimes, especially when he had time on the clock, but his mind could only take so much thinking, and time was catching up, so for a great many number of moves he thought locally.

AlphaGo doesn't tire and thinks globally on every move.

We saw that time and time again. Lee playing locally and AlphaGo surprising everyone with an unexpected move somewhere else on the board.

Obviously it is better to be globally optimal than locally optimal.

highwind• 10 years ago

Doesn't principle of optimality state that if a solution is optimal than any of its sub problem must be optimal as well?

wickawic• 10 years ago

Instead of downvoting this comment, let me provide a counter example. Imagine the knapsack problem, where you can lift 10kg and you have a 10kg gold bar and 5kg cinderblocks. In this case solving 2 5kg knapsack problems will give you two cinderblocks, which is obviously not optimal.

I don't know a lot about the strategy of go, but it seems to me that any play that doesn't take into account the entire state of the board is allowing for the same class of sub-optimal behavior as the example above.

pixl975• 10 years ago

Imagine a condition where the local optima and global optima are not aligned.

quietplatypus2• 10 years ago

Problem: One person remaining on island, 2 people left.

Optimal solution: Kill yourself.

Sub-optimal solution: Both live.

sp3322• 10 years ago

But in Go you can only make one move at a time. So even if a move is optimal for one part of the board, there might be a different move that you should have made first.

leeoniya4• 10 years ago

Obviously it is better to be globally optimal than locally optimal.

the implication of having machines dictate our behavior [WRT environment, each other] to optimize the survival of humans far into the future seems a bit closer to reality.

asdfologist6• 10 years ago

I don't want to downplay the amazing breakthrough that AlphaGo represents, but I think it's a bit melodramatic to extrapolate from a program winning a board game to AI's impact on humanity's survival.

kibwen• 10 years ago

"Optimization" presumes the existence of concrete criteria by which an outcome may be judged, and often a timeframe within which to judge it (e.g. if I want to maximize my wealth when I retire I should fully fund my 401(k), but if I want to maximize my wealth next year then I shouldn't fund it at all). A board game has unambiguous criteria for victory; human civilization, not so much.

kotach3• 10 years ago

AlphaGo is approximating global optimality by finding local optimality. Local optimality is already computationally very hard, but it is exactly what AlphaGo is doing.

The rollouts they are doing, evaluating every probable move, it is a search process of trying to find local optimality. You have a current state of the board and you're trying to find a decision that minimizes your future regret. It is by definition local optimality.

Global optimality would be finding a sequence of moves that wins you a game.

robotresearcher3• 10 years ago

Global optimality would be finding a sequence of moves that wins you a game.

Is it known that Go is winnable? It may be that only a draw could be guaranteed. A globally optimal player would be provably able to do (whichever of these it turns out to be) for any legal position. This is harder than winning a particular game.

robotresearcher3• 10 years ago

For reference, AlphaGo is not globally optimal. Go is much too big for that.

(edit: also there'd be no point playing a game against it!)

Radim• 10 years ago

I believe the OP refers to global/local as they are used in Go. There, global/local play is a concept taken for granted by humans, with explicit names like "sente" and "tenuki".

I think the OP simply implies AlphaGo doesn't seem bound by these human cognitive crutches.

This is not the same as "global" in the theoretical sense, as in "finding the perfect solution / exhaustive proof". See also "game temperature" [1]

[1] http://senseis.xmp.net/?Temperature

mchahn• 10 years ago

It was fascinating when the author experienced a weird bit of nausea when thinking about the implications of AI. I'm not sure how I feel about it yet. It does scare me a bit.

agumonkey3• 10 years ago

Think about that we're constantly pushing toward our own replacement.

catshirt• 10 years ago

considering the possibility of augmentation, supplantation seems like somewhat of an assumption.

mdpopescu• 10 years ago

When a student surpasses the teacher, that's supposed to be a good thing.

david-given• 10 years ago

Don't we normally call that having children?

ktRolster• 10 years ago

I saw this tweet earlier:

In 13 years, computers progressed from beating Chess to mastering Go. I hope that in 13 more they will learn to not steal focus while I type

71817188• 10 years ago

Did Lee Sedol have a reasonable idea of AlphaGo's recent strength? To put it more harshly, did DeepMind misrepresent the true strength of the program?

I find it very hard to find information about this online.

lukaslalinsky• 10 years ago

They challenged a top player for $1M. Doesn't that imply they think it's stronger than any human player?

gus_massa2• 10 years ago

I think they expected that the computer to play a good game, but probably they were not sure about the outcome. Anyway, Google has Billons in the bank, so if they were defeated they would not miss that single Million too much.

wolfgke2• 10 years ago

This can also imply that for Google it is worth $1M to know where AlphaGo still has weaknesses.

Bud• 10 years ago

It implies that it costs money to get a high-profile match against someone like Lee Sedol, and it costs money to draw public attention, and it implies that they thought they at least had a decent chance. Nothing more.

fma7• 10 years ago

How much money do companies spend on Super Bowl ad for a 30 seconds of publicity.

How much publicity has Google received for $1 million? (Much less than that actually, since they won)

tim3339• 10 years ago

The $1M gets paid out by Google whether AlphaGo wins or loses. I just goes to charity if it wins.

lambda2• 10 years ago

No, they did not misrepresent the strength of the program. They did not know the strength of the program. The purpose of the match with Lee Sedol was to determine that.

So far, they have tested a previous iteration of the program against a much lower level professional, as well as all of the other Go engines they could test it against. It easily defeated all other Go AIs, but no previous Go AI has been able to compete at a professional level. It also defeated Fan Hui, the lower level professional, though it made several mistakes and lost a couple of the informal games (with lower time limits) it played against.

Since then, they've had several months to tweak it, and then let it train against itself some more. The thing is, to really determine its strength, you need to have it play serious games against professional players. So there really was no good way for them to determine its strength without playing these games. It's possible that it only would have improved a little since it's games with Fan Hui; in that case, the match with Lee Sedol may have been easy for him. However, it appears that it has improved substantially.

return04• 10 years ago

He knows it beat the european champion 5-0, and that it outperforms all commercial and free Go games.

pixl975• 10 years ago

Which was meaningless at the time, and I don't think he knew that game beat the European champ until shortly before his game. The difference between the two champions was so large that Lee would be expected to win 95% of the time. All other Go AI was exceptionally weak when compared to the top level players.

jonknee4• 10 years ago

If he didn't before game one he definitely did afterwards. I have a hard time believing it would be easy or that he didn't try as hard as he could.

petegrif• 10 years ago

Holy crap. It's a total beatdown.

jacquesm• 10 years ago

So far, but yes, it looks bleak (or good, depending on which side you're rooting for).

guelo• 10 years ago

I don't understand how this is considered fair. AlphaGo has been trained on a database that includes every recorded game Sedol has ever played while Sedol is seeing AlphaGo's play style for the first time. Sedol should have been allowed to play against AlphaGo for a few months before the match so he could study its style.

krackers• 10 years ago

One member of the deepmind team responded to this very question during the interview at the beginning of part 3.

He said that the training data set size is much, much larger than the number of lee sedol games. It is like a drop in the ocean and not enough to significantly influence the resulting policy network.

gus_massa2• 10 years ago

Perhaps the computer didn't know that it was playing against Lee Sedol, but from Wikipedia "As of February 2016, he ranks second in international titles (18), behind only Lee Chang-ho (21)."

I don't know the details of the algorithm and perhaps it doesn't give more explicit weight to his games. But I wouldn't be surprised if after some iterations the algorithm decided to give more implicit weight to the games of the world leaders.

yeukhon2• 10 years ago

I did not downvote you, FYI.

But I think a computer as a training buddy is indeed a good idea to improve Lee's skill. I don't know how Lee feel right now, he's defeated, there must be an enormous pressure inside of him, but I think he will appreciate the challenge because he needs more challenge! He has played against top players all over the world now for his 20+ years since earning 1 dan rank. Also, I think the computer play against itself and many random moves. Unlike human players, one would expect computers to play unconventionally since the computer can predict so far ahead about the probability of maximizing winning.

fma7• 10 years ago

I wouldn't say Lee is unchallenged. Ke Jie defeated Lee just before Lee played AlphaGo.

But I would say he might feel like crap having lost his title to a 19 year old, and now losing to a machine all in the same week.

clarle• 10 years ago

This is more than fair.

Go AIs weren't expected to reach this level for at least another 10 years.

Before AlphaGo, Zen and Crazy Stone (the previous Go AIs) could only play against top-level professionals with a significant 4-5 stone starting handicap, and this was less than 3 years ago. A 4-5 stone handicap is basically taking control of half the board before the game has even started.

It really shows how the neural network approach made a huge difference in such a short time.

emcq3• 10 years ago

Part of this timing jump is Google throwing hardware at the problem with a large 280 GPU + 1920 CPU cluster. I would venture this is almost 100x bigger than most of the Go AI hardware we've seen to date. The nature paper suggests without this cluster it would be playing competitively with other single workstation Go AI, but nowhere near top level players.

zeroxfe• 10 years ago

It's probably not that different from some new hotshot kid that has no recorded history of playing, but got really good playing on the street and studying the masters. Kid gets discovered by some credible major promoter who challenges Sedol to a $1M game.

randomguy7788• 10 years ago

sedol's game actually has zero bias over how alphago plays. the team lead for alphago himself said that its like a drop in the ocean/bucket for alphago.

asdfologist6• 10 years ago

It's actually entirely possible that if the program were unsupervised, i.e. had to learn Go "from scratch" without relying on any human games, it would be even stronger than it is now.

arcanus• 10 years ago

I've very, very suspicious of this statement. Unsupervised learning is generally speaking not as advanced as supervised.

While unsupervised learning might be the holy grail, this victory is really about deep learning, which is an advanced supervised learning technique.

jonknee4• 10 years ago

Deepmind is going to work on that next (according to one of the staff interviewed on the broadcast). It will be interesting to see if other playing styles develop.

plank• 10 years ago

So what? I could watch every game Roger Federer played in any tennis tournament and still lose all sets to love. It used to be that computers could only do combinatorics better, but that where 'intuition' played a strong part, there was still hope for us humans... Well, guess I will have to start playing Calvinball...

philovivero• 10 years ago

Okay, I have $10k says Plank beats the computer in Calvinball until at least 2020. At that point, I'm only willing to bet $1k/game.

fma7• 10 years ago

There's a difference between having skills and having knowledge.

In your case you have knowledge but lack skills.

When Federer was losing to Nadal he changed his practice and game to counter Nadal's style. He took knowledge and applied it to his skills.

When Federer came out with SABR people were like wtf. Now they know of it and put more umph on their second serve. See. Knowledge + skills

fma7• 10 years ago

Also assuming you are a tennis player...if you do not study your opponents' shots during warm up, you are doing it wrong.

The way the would play against a lefty is different than a righty. Someone with lots of topspin vs someone who hits flat, a pusher vs power hitter, etc.

_yosefk• 10 years ago

As it happened in chess, machines will beat humans in Go no matter how much knowledge about their inner workings will be provided to the human. (I've watched this story unfold in chess and the various hopes of how humans are still somehow better. $1K sez Go is exactly the same story. Can't beat a machine in a formal universe with a defined goal.)

pixl975• 10 years ago

Can't beat a machine in a formal universe with a defined goal.

That's why the next goal AI developers are pushing towards is incomplete information games.

hyperpape2• 10 years ago

I don't think this is unfair, but I think other people are replying to a suggestion that AlphaGo might be optimized to beat Lee that I don't see in the parent comment.

What does seem right is that whatever strategic insights are in Lee's play are reflected in current games--his and the younger generation who came up in his shadow. Whatever strategic novelties shape AlphaGo, they are totally new to Lee.

I don't think that would make the difference: 1) there's no trick to be learned and 2) the same thing happens with human players to some extent--when Lee Changho appeared on the international go scene, his style was misunderstood and underestimated, even when his games were public.

However, it is true that there is a real asymmetry--AlphaGo may not know Lee from other players, but it has had the opportunity to "study" the best games of the current players, and no one outside of DeepMind has had an opportunity to study its games.

212• 10 years ago

Lee Sedol didn't complain about this. In fact, even a few days before the match Lee Sedol was predicting a 5-0 win for him.

If someone is clearly better than you, you can study it's style all you want, it won't make a difference. You probably wouldn't even understand it's style.

andromeduck2• 10 years ago

But that wasn't based on the performance of AG but sumlsr AIs

fma7• 10 years ago

That's a very amateurish and closed minded comment.

There's a reason professional sports team and players study the styles of their opponent. Teams employ statisticians for this exact role. Before games, teams study the style of their next opponent. After every after game teams go and study the recordings of their last game.

est• 10 years ago

Sedol should have been allowed to play against AlphaGo for a few months

It's not whether AI can outperform human, it's a matter of when.

sdenton43• 10 years ago

It would be interesting to know AlphaGo's estimated probability of winning as the game progressed; presumably we can see directly how worried it was at any given point in the game. And thus get another sense of whether it ever really felt threatened by Sedol.

At this point, the match is won, but games 4 and 5 will commence. The question shifts from whether AlphaGo is better than humanity's best, to whether humanity can even have a chance of beating AlphaGo in a single game. And so far, it sounds like the answer is likely to be a resounding no.

teraflop• 10 years ago

This isn't exactly what you were asking for, but here are some graphs from a different Go AI (Facebook's "darkforest") showing its estimated win probabilities: http://zhuanlan.zhihu.com/yuandong/20639694

sdenton43• 10 years ago

Neat! My understanding is that AlphaGo is a stronger player, and so will likely have different estimations than the darkforest. But this is still really cool.

EGreg• 10 years ago

Can anyone with Go knowledge comment on this?

raverbashing• 10 years ago

This is different from DeepBlue and Garry Kasparov

Chess may have a big search space, but it is "well behaved". Moves might be out of the ordinary, but not too much (weird moves that still help to win the match are rare)

AlphaGo might also have learned from past matches, but that doesn't give all the answers

The black-box aspect, the fact that you can't understand what it is thinking is curious, to say the least

asdfologist6• 10 years ago

I don't know if that's true. "Computer moves", referring to weird moves that computers play that often turn out to be very good, tend to get mentioned a lot in chess. And you might hear about them less and less, because many of them have been absorbed into the human chess repertoire.

jdietrich• 10 years ago

Also notable is the verb "spacebar", meaning "to play the same move an AI would". The chess world provides an interesting microcosm of how humans adapt to a world dominated by AI.

seanwilson5• 10 years ago

The black-box aspect, the fact that you can't understand what it is thinking is curious, to say the least

How true is this? Presumably you could ask AlphaGo to give you a list of the top 10 best moves it considered and perhaps, given a certain move, what it sees the board looking like 10 moves ahead.

If you're watching the commentary, one of the commentators (Michael Redmond) will try to explain the outcome of certain moves and this would seem to be equivalent although I imagine he could explain this in terms of the endgame better.

clickok2• 10 years ago

How strong is AlphaGo?

DeepMind doesn't generally comment on these sorts of things, but I think they knew it would be strong, just not how strong.

It is hard to accurately gauge the playing strength of such a program for the following reasons:

* Go is not solved, it doesn't really even admit a good heuristic for close games. In chess, given enough time, we can do a pretty good job of searching the possible outcomes from a given position, and even if we can't evaluate all the possible endgames, evaluation functions exist that give us an idea of which side is ahead when we terminate the search. Go has a much higher branching factor (more moves available at each turn, making the search more expensive) and short of a catastrophic blunder, it's hard to quantify who is ahead at each point in time[1]. So we can't (in general) quantify optimal play, and therefore cannot quantify how much AlphaGo (or anyone else) deviates from it.

* One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning. According to some people I've talked to, this can be suboptimal, but there are methods of addressing this[2]. Note that you can't just change the objective function from "winning" to "win by as much as possible" without potentially reducing AlphaGo's strength.

* The value function learned by a deep net is hard to interpret, partly because it encodes information about the possible futures arising from a position, and partly because it involves a tremendous number of calculations to compute. We do not know what it is representing at the intermediate levels-- techniques that aim to visualize or cluster unit activations can provide a bit of insight, but there's always the possibility that we're interpreting the patterns incorrectly because we're trying to fit it into the framework of "what would a human think?". Further, the representation is somewhat monolithic-- we can't tweak the value of one thing without changing the values of others. In chess engines, we might modify the material value of, say, a knight, without affecting the value of a rook. In a convolutional net, if we adjust the value of one position, it will tend to affect the value of many others.

* We can attempt to quantify its strength by comparing it to other programs, but AlphaGo has already crushed other programs (99% win rate), so all that it tells us is that it is stronger than those programs.

Essentially, without the ability to perform a significant amount of searching, or gauge strength via margin-of-victory, or to examine the program from other angles, it's hard to gauge just how good the program is.

The only thing we can do is throw skilful opponents at it, and see if it fails eventually[3]. In its current incarnation, though, it seems like its playing strength is just going to be "stronger than you".

---

1. Hence the need for Deep Reinforcement Learning-- we learn how valuable each position is based on the results of the positions that it can lead to.

2. By modifying the search to maintain a given margin once it has been attained-- but I do not work at DeepMind, nor am I an expert on MCTS like some of my colleagues, so I don't know if this could adversely affect AlphaGo's overall strength.

3. We might be able to get a better idea by having extremely strong players play against weaker versions of AlphaGo (ones with less computing power) and then sort of telescope upwards once a good baseline has been set, but it remains to be seen if the current level of invincibility is due to it not having been around long enough for its weaknesses to become apparent[4].

4. How would an AI researcher play against it? I've talked to a few people, and the answers have been: (a) play the game to the conclusion, don't resign; (b) attempt to take the game off-the-rails into parts of the state-space it hasn't really explored before (although this is risky because the human player is more likely to make mistakes in these cases as well), and (c) let it get ahead somewhat so that it "relaxes" (see the points about margin), allowing you to catch up and overtake it.

cottonseed• 10 years ago

The way to find out how strong AlphaGo is is to start giving handicap and see when the game becomes even. A difference in rank of 1 stone corresponds roughly to a 2:1 advantage. It is traditional in long-running matches to increase the handicap by 1 after one player wins three consecutive games. I haven't heard any talk of that here -- I'm sure Lee Sedol wants the chance to beat AlphaGo in an even game -- but it would be interesting. It would also be interesting to see how AG plays when it thinks it is behind.

clickok2• 10 years ago

That might work; the concern is that this would take it too far off of the task it was trained on. That is, if it doesn't have a lot of experience being significantly down, then it won't play nearly as well when trying to catch up-- but that doesn't matter in even games because it never gets that far behind.

You're right that it would be interesting to see, though-- we need to get better at understanding these sorts of systems, at least until they can start optimizing themselves.

Alternatively, we might train a different agent (OmegaGo?) to try to win by the largest margin possible-- if it works as well as AlphaGo, then that might give us some more insight into how strong both programs are.

ghaff3• 10 years ago

One known aspect of programs using Monte Carlo Tree Search is that they play to win, and are willing to sacrifice margin of victory to maintain or increase their odds of winning.

One interesting implication of this is that it very much applies to perfect information games--not games that involve an element of luck or, indeed, pretty much anything that involves the real world. In those latter environments, you absolutely want to run the score up or otherwise take small chances to gain an immediate advantage to create a cushion against future unexpected events (even if they slightly hurt your odds if those events don't occur).

bsder2• 10 years ago

are willing to sacrifice margin of victory to maintain or increase their odds of winning. According to some people I've talked to, this can be suboptimal,

I suspect this is less suboptimal when a computer is involved since a computer never gets tired or makes an attnetion-based mistake.

eagsalazar2• 10 years ago

I'm curious if AlphaGo is simply winning by virtue of having more computational capability (and therefore can never be defeated consistently by humans) or if in its training it actually discovered and is now deploying interesting new tactics that humans, studying these games, can uncover and therefore use to defeat AlphaGo in the future.

hodwik3• 10 years ago

Sedol said that it was doing tactics he had never seen before -- like the now infamous move 37 that caused Sedol to get up and walk out of the room.

cjbprime3• 10 years ago

Apocryphal, but romantic -- he left the room ten seconds earlier.

partycoder10• 10 years ago

move 37 was a black move (lee sedol)

return04• 10 years ago

AlphaGo was first trained with human data only to create an initial "predict the next move" network. Afterwards, they let the network play with itself to learn how to win. So it's quite possible that it has its own tactics.

nzonbi7• 10 years ago

This is awesome, finally AlphaGo has beaten Lee Sedol. Congratulations to the team at DeepMind and Google. But I will go straight to the really interesting point: My completely baseless intuition tells me that we will have full AGI (artificial general intelligence) in 5-10 years. Maybe not very powerful at first, but a least working. Real general intelligence. I say this as a programmer with superficial knowledge on ML (machine learning). All that is needed is to figure a way, to make neural networks handle the flow of thoughts of human consciousness. Reasoning, short/long term memory, basic controlling "emotions", etc. Then to assemble this central thought unit, in a system with various neural networks to handle more specialized tasks: Vision, natural language processing, audition, motion,etc. The task is of course extremely difficult, but I have the feeling that it will be possible to overcome.

Once this have been achieved, it is downhill from there. The singularity. In my opinion, we must start to consider how this tremendous breakthrough will affect the human race. Our lives, economy, culture, etc. Awesome things we will be witnessing in the coming times. It would be great if it were possible to make public, the progress made researching specifically AGI. I wonder how big of an effort is currently being done. This would be possibly the biggest discovery ever for humanity. It deserves a fantastic effort.

bgar• 10 years ago

I don't know if you're serious, but how can you make such grand predictions, considering you yourself admit to having only superficial knowledge of machine learning?

nzonbi7• 10 years ago

It is just a guess. Human thought, although highly complex, is in my opinion something actually feasible to be simulated. If I had a large research budget under my control, I would be betting most of it on AGI.

ilaksh• 10 years ago

No one can really predict the future. But given the number of times AI pessimists have been proven wrong in recent years and the massive increase in AI investment, my best guess is that truly human-level artificial general intelligence is in fact around the corner.

Considering what a large impact this will have, we must take that possibility seriously. Not to do so would be foolish.

AIs can play chess, recognize speech consistently, drive cars, win at Jeopardy and now Go. Deep Mind has some general capabilities. We must anticipate programs with even more general intelligence.

edavison1• 10 years ago

"All that is needed is to figure a way, to make neural networks handle the flow of thoughts of human consciousness."

Good luck with that!

smitherfield• 10 years ago

I just thought of an interesting (and somewhat more "real-world") task for AI research: Can an AI outperform a human at play-calling in [American] football?

cortesoft• 10 years ago

The problem is that it REALLY hard to determine who "wins" at play calling.

rcthompson• 10 years ago

Considering that plays are often called or changed by quarterbacks on the field right before the snap based on what the QB sees from the opposing team, I think it would be difficult to provide all that input to a computer AI.

jonknee4• 10 years ago

A neural net would be able to read the opposing defense more quickly and accurately than any human, but it would not be a fair fight (the human doesn't get a bird's eye view of things!).

Harimwakairi• 10 years ago

Yahoo's fantasy football software creates millions of machine-written summaries of their fantasy football matches after each week of play. They're several paragraphs long and point out interesting things about the particulars of the teams' performance.

I would imagine that given the "All-22" camera footage from a football game, it would be possible to construct a piece of software to monitor what was going on (visually identify players by name/number, determine who has the ball, etc.) and then convert that model to a stream of English text.

Making it nuanced and entertaining, on the other hand, would be a challenge. :)

bsder2• 10 years ago

Some of the coaches already do this. Their "card" includes plays that say "In situation X, the probabilities say to call play Y."

The problem is small sample size. You only have the plays as called in the games to rely on. You can't "replay" the game with a different set of plays and see what the result would be.

This was a problem even in Moneyball. It's just that On-Base Percentage has a really good correlation to runs and therefore wins.

131hn• 10 years ago

Can alphago be defeated ?

iainmerrick2• 10 years ago

Deep Blue can be defeated at chess now.

tim3339• 10 years ago

Facebook are working on a rival AI.

partycoder10• 10 years ago

Skeptics have said that achieving this milestone would not happen within this decade, our lifetimes or even ever. It happened yesterday.

It took Lee Sedol many decades of his life to train to achieve this level. And his ability to pass on his skills is limited. Now that a computer has achieved this level, the state of the neural network behind it can be serialized and ran into an unlimited number of computers and have millions of systems that are more proficient at Go than the best player in the world.

People have said that achieving the cognitive level of the human brain requires to match its computational power. But if you take out all the parasympathetic and motor boilerplate, what is actually left for mental tasks is much less from that, most of that power is not even recruited for higher level mental tasks. That lowers the bar for strong AI.

Then, strong AI can be immortal, and never physically deteriorate from aging. Strong AI can multiply infinitely and communicate at a rate that would be equivalent to writing millions of books in a second. It could transfer all its knowledge in seconds. It can also recursively improve itself. This advantage will lower the bar for strong AI even more.

doktrin• 10 years ago

Skeptics have said that achieving this milestone would not happen within this decade, our lifetimes or even ever.

Did any credible skeptics actually claim this would never happen? If so, I'd love to hear their reasoning.

partycoder10• 10 years ago

Enough people is skeptical about AI to make it a problem.

Now, being more specific about Go:

- Lee Sedol himself was confident he would win.

- Ke Jie said he would win against AlphaGo. I think he would lose too.

ghaff3• 10 years ago

Certainly a lot of people ten years ago would have bet against this happening by 2016--especially before Monte Carlo approaches were tried. But I doubt you would have found many takers for betting against it happening in a 100 years (much less it never happening).

Autonomous vehicles are similar. IMO, it would be a fool's bet to bet against them ever happening. But it's reasonable to debate whether they'll be mainstream Level 4 consumer devices in 10 years, 25 years, or 50 years.

VikingCoder2• 10 years ago

A neural network beating a top human at Go? Cripes, if Marvin Minsky didn't say it would never happen, I'd be stunned.

return04• 10 years ago

On the other hand i would like to read an assessment of the significance of this advance. While AI may have perfect applicability to these perfect information games, we may still be decades from useful real world generic applications.

partycoder10• 10 years ago

It is good enough now to replace millions of jobs. Transportation, retail, press, trading, etc.

It is also good enough to put it in a combat drone that visually recognizes objectives and people. This can be all commanded by a small group of people, anonymously, and detached from accountability and without casualties.

nzonbi7• 10 years ago

In my opinion a possible path is to make some sort of Symbolic AI (https://en.wikipedia.org/wiki/Symbolic_artificial_intelligen...) system, and put machine learning on top, and even below of it. You could start with a very basic Symbolic AI, easier to understand. Then growth it gradually. The cool thing of this approach, would be that you could inspect the symbolic system to get a glance at how the AI system is thinking. Obviously a purely neuronal system can also do the trick. That is how the human brain works. But it would be great to make a system based on a symbolic AI, because it could be very interesting to observe its inner working. And it would also be possible to interact with it at the symbolic level.

aerovistae2• 10 years ago

Exactly so. While this is a major advance, I roll my eyes at the sea of comments hailing the arrival of a superior intelligence.

When the AI is cognizant of the fact that Go is a game, and knows what a game is, and perceives that the game is taking place in a larger reality, where there are other things happening while the game is going on....then I'll be impressed.

jtolmar5• 10 years ago

AlphaGo's algorithms will generalize easily to other perfect information games, including ones that include randomness like Backgammon. Games with imperfect information will take some more advances, but AlphaGo just opened up a lot of interesting potential paths, so there's a decent chance someone pursuing one of those will find something quickly.

The sheer amount of computing power that AlphaGo needs will be a problem for games as complicated as Go for people who aren't Google. There will probably be more activity in simpler games with interesting properties.

Non-game applications will require a fully formalized set of rules. The fact that formalizing what you want is hard is the reason we need programmers in the first place, so nothing fundamentally different there.

andreyk• 10 years ago

I think this is a great (and very overlooked) in depth assessment of the significance of this advance http://www.milesbrundage.com/blog-posts/alphago-and-ai-progr...

I do feel that given all the hype and speculation about strong AI researchers in Deep Learning should really come forward to contextualize the real impact of this - which I think is not all that great.

tim3339• 10 years ago

Part of the significance is that it was not the main goal for Deep Mind and Hassabis. He's said his main goal is to solve intelligence and it approaching it by trying to reverse engineer the human brain, having done a PhD in neuroscience. Hippocampus next. At this rate I'd be surprised if it took decades to real world generic applications.

seiji• 10 years ago

strong AI can be immortal

My grandfather was very racist and never changed as long as he lived. I'm glad most people aren't immortal. There's no guarantee a full AI wouldn't be tempted by evil and just become republican (or worse, a VC).

It can also recursively improve itself.

So can people (the more you know, the more you can learn), but most don't. It's important to remember "intelligence" isn't an abstract concept—intelligence is also embodied in personality—and personalities have wishes and goals and desires and loves and hates and that one song they can't get out of their head. A true "strong AI" will be fully conscious, not just algorithmic function bating.

Good luck telling a mildly strong godform to stop tripping on youtube videos and instead solve the global economic stability equation over lunch.

This advantage will lower the bar for strong AI even more.

That's kinda foofy conjecture. Being good at rectangular grid outcomes isn't necessarily a step in any direction towards a hands-off tax evaluating robot.

It feels really really good to talk about how AI will be a hundred billion trillion times smarter than the combined brainpower of all humans that have ever lived, but it feels good in the same way thinking dead people live again after they die feels good—it triggers that warm wishful thinking parietal lobe that removes a bit of reason for the sake of an overarching calmness.

Enthusiasm is great, but tempering with real expectations and less technopriesthood is better.

nzonbi7• 10 years ago

In the grand scheme of things, what is the point of humans repeating billion of times the same cycle?: born, grow, learn, achieve some things, have some fun, and die. It is awesome a few times. But it is worth to repeat and repeat that for ever? Not if there is a much better option: The singularity. It has the potential to enhance the human race in formidable ways. Yes,there is the risk that the AI may chose to destroy us. But in my opinion is a bet that we should take. Just try our best to not be destroyed in the way. But we shouldn't leave this opportunity unexplored. It would be the biggest achievement of humanity.

quietplatypus2• 10 years ago

Pretty much this.

partycoder10• 10 years ago

You can recruit neurons within your own brain, not plug the equivalent of 4 brains into your brain and increase your cognitive abilities.

Now, while many people die and their ideas die with them, there are built in aspects of our brain that are hardwired. People eat animals because they are tasty, people follow the life of Kim Kardashian and waste money buying a rolex and wear fur just because the stupid way our social functions are hardwired in our brain.

aprescott2• 10 years ago

It's worth remembering that AlphaGo was partially, initially trained on the previous game records of professional players, to form the basis of its policy network and so on. AlphaGo's victory over Lee Sedol is impressive, but it does rest on the combined total experience and long study of humans. That's why it's even more exciting to me to hear that one of the DeepMind team's next efforts will be to see how AlphaGo plays when it starts learning "from scratch".

keehun• 10 years ago

I agree with the excitement. A possible outcome is that the AlphaGo trained "from scratch" will not be as strong--but another possibility is that by eschewing any bias for human tradition, perhaps it will come up with even better strategies that no one has thought of before.

seanwilson5• 10 years ago

Can anyone elaborate on how they would teach it from scratch? Would this mean only giving it the ability to play valid moves (and against itself) and give it access to the final score?

nickpsecurity• 10 years ago

This was my argument against the StarCraft claim. If it wins, it will likely have just imitated human pros' thousands to millions of games like a smarter version of old chatterbots.

Whereas, good human players learned as they went with far fewer matches, trial and error, planning, and learning from pto games sometimes. We can even go from Age of Empires to Starcraft with little prrparation and still do OK.

the_af3• 10 years ago

I understand very little of Go beyond its basic rules and having played only a couple of matches with a friend who knows as little as I do...

I reviewed the match, and it seems that very quickly White (AlphaGo) went aggressive, and Black was trying to contain it. I suppose a human player can commit serious mistakes by playing too aggressively from the start, right? But the summary of the game speculates move 31 might have been the losing move (maybe implying the game until that moment wasn't going so bad for Lee Sedol?), while to my untrained eyes it looks as if White was constantly on the offensive from the start, and Black was playing almost exclusively to contain it.

Or am I reading this wrong?

partycoder10• 10 years ago

In game 3, when Lee Sedol played on the left side, a capture race started. AlphaGo made his dragon look vulnerable to be captured. Lee Sedol attacked it, and AlphaGo defended it enough so it would not die, but making a huge territory on the bottom part of the board. When AlphaGo's dragon lived, Lee Sedol realized AlphaGo was ahead in score, and needed to invade the largest AlphaGo's territory in the bottom. This invasion was not feasible, and Lee Sedol was relying on a long shot strategy involving kos. This idea was finally refuted by AlphaGo, leaving him with no feasible ko threats, forcing Lee Sedol to resign.

autarch• 10 years ago

I think that most high-level Go players (including AlphaGo) play moves that serve multiple purposes simultaneously. The ideal move reinforces your own position while forcing your opponent to defend their own.

White certainly played plenty of moves that required black to respond, but black also did the same. For example, the sequence beginning at 77 is black getting a foothold inside an area that white might've hoped to claim later. Then at move 115 black attempts an invasion into a very strong white area, ultimately failing to escape or live, leading to resignation.

That all said, I wouldn't say that either player was particularly aggressive. I've watched plenty of human games where one player or the other very intentionally picks an all-out fight. If you're really good at reading ahead locally and you think your opponent is not, this make sense. This is especially true if your large-scale game play is not as good as your tactical play.

This is also common in high handicap games. If white can't slowly eke out an advantage early on, the only remaining tactic might be to go for a huge fight and see what happens.

lilcarlyung• 10 years ago

Well, at least we still got poker. Until AI knows how bluff or how to call out bluffers.

VikingCoder2• 10 years ago

Hmmm...

http://www.latimes.com/nation/great-reads/la-na-c1-claudico-...

http://spectrum.ieee.org/tech-talk/computing/software/comput...

http://www.npr.org/sections/alltechconsidered/2015/01/08/375...

Sir_Cmpwn2• 10 years ago

What are some good resources to learn Go?

dyoo1979• 10 years ago

There's a good interactive tutorial at http://playgo.to/iwtg/en/

The community has also collected a lot of good Go resources at Sensei's Library: http://senseis.xmp.net/

Good luck!

Sir_Cmpwn2• 10 years ago

Thanks!

andrewflnr• 10 years ago

What I found especially chilling was how the AI took its time when it thought it was ahead, and played exactly as aggressively as needed to win. It seems like we're seeing the full benefit of detaching logic from human emotional drives like pride, loss of nerve, etc. Which at first sight looks awesome, but in a way it's deeply terrifying, because if such a machine is ever given power in the real world, we would really like it to have some regard for human emotion. More than anything else I've seen, this makes me feel in my gut like I'm seeing my replacement.

jtolmar5• 10 years ago

On the perhaps more reassuring side, it's possible for there to be an AI that takes into account human emotion, but does not have any of its own. Given power in the real world, it would do a better job of making people happy than a human with all their biases. It wouldn't assume that other people must like things it like, for example.

(Of course, making an AI that understands emotion isn't easy. And happiness by itself is an insufficient goal if we're building ourselves a benevolent overlord.)

armitron3• 10 years ago

I view all such arguments about "friendly AI", "emotion", having machine intelligence "understand us" as wishful thinking at best and laughable delusions otherwise.

I think the writing on the wall is clear, to anyone who cares to take a look. The moment we create an artificial intelligence capable of self-improvement and let it loose, we will have fulfilled our function (others say destiny) and will therefore be obsolete in every sense of the word.

What will happen to humanity after that point is irrelevant.

You don't see humans trying to keep bacteria "in the loop", why expect otherwise from our artificial progeny?

mikeash4• 10 years ago

Such an AI would also do a great job of making people sad, or terrified, or numb, depending on just what motivated it. I don't find that reassuring.

aprescott2• 10 years ago

the AI took its time when it thought it was ahead, and played exactly as aggressively as needed to win

Don't human professional players do exactly this? They care about playing the best move and winning. The difference is that AlphaGo is likely much more accurate at determining what "95% probability of winning" means in terms of gameplay. A human has a harder time judging the eventual outcome of a game, and so plays more "aggressively" (favouring point margins to account for variance in the estimate, etc.) than AlphaGo might.

bobbles• 10 years ago

So what happens if we play AlphaGo vs AlphaGo?

tim3339• 10 years ago

They did loads of that in the cloud during training.

mingodad• 10 years ago

Out of curiosity how much electric power is alphago burning while playing ? Does someone know it ?

hutzlibu3• 10 years ago

Much more than then "stupid" human used, I guess. So for now, we are at least much more energy-efficient ...

tim3339• 10 years ago

Apparently 1920 cpu's so maybe 200 kW?

naveen99• 10 years ago

Seems like just yesterday (actually seven months ago), I was saying dbn's aren't good at go yet in an argument: https://news.ycombinator.com/item?id=10028908

i guess it's time to play with dbn's again. At least I have accumulated a few more Titan x cards in the mean time.

hyperpallium• 10 years ago

Huh, Go doesn't require intelligence after all.

botw• 10 years ago

The actual match is largely effected by psycology at the time of game. The match is somewhat unfair to human player. AlphaGo and its team know all about each human player and preference. But human player knows little about AlphaGo. Also it is AlphaGo team who decides whom to choose to play against AlphaGo, not the other round. In the sense of psycology and , the game is greatly favor to AlphaGo right now. This is the difference with the match between IBM DeepBlue and the chess world champion 10 years ago. It should be that any player(human or AI) can be allowed to enter the game, with or without a fee.

AlphaGo shows its true strength in 3rd victory against Lee Sedol

Related Stories