AlphaGo beats Lee Sedol 3-0 [video]

https://www.youtube.com/watch?v=qUAmTYHEyM8

by Fede_V • 10 years ago

566 404 10 years ago

youtube.com

niuzeta• 10 years ago

Impressive work by Google research team. I'm both impressed and scared.

This is our Deep Blue moment folks. a history is made.

yulunli• 10 years ago

This seems to be more impressive than the Deep Blue moment. In 1996, Deep Blue didn't make it on the first try. Even in 1997, it has been a draw until the 6th game. Although AlphaGo has 2 games to go, the first three seem to be a clear victory.

partycoder8• 10 years ago

Far more impressive than the Deep Blue moment.

This time the computer did not win out of pure bruteforce. Deep Blue relied on an opening book and massive computational power to explore the game tree. After the opening it was pretty much on its own, bruteforcing moves.

This technology used a neural network trained with hundreds of thousands of games which provided the pattern matching aspect, combined with the bruteforce move sequence reading, the montecarlo tree search... and 1200 CPUs + 600 GPUs.

esturk• 10 years ago

Give credit where credit is due. This is DeepMind's research, and Google acquired them in 2014. Of course, Google gave them a lot of resources to train AlphaGo. But let's not bury the subsidiary, or I guess they would both be subs to Alphabet now making the two even less related.

gniv• 10 years ago

DeepMind is part of Google.

bluecmd• 10 years ago

Well, 2 of the authors are from Google itself, so it's not fair to say it's 100% DeepMind as well.

drjesusphd• 10 years ago

If they wanted ultimate credit, they shouldn't have sold. They made a judgment call and part of that is Google gets a share of the congratulations.

timothya• 10 years ago

According to the pre-match interview[0], the AlphaGo project is about two years old, and started out as a collaboration between two members of DeepMind and a member of the Google Brain team (Google's deep learning team which pre-dates the DeepMind acquisition by several years, according to Wikipedia). So really, AlphaGo is a combined effort that was always a part of Google, and I think it's pretty clear that resources provided by Google (money, equipment, and tooling) in addition to the techniques developed by both Google and DeepMind made this project a reality. I think they both deserve a lot of credit, certainly, but I wouldn't say that the credit is all that misplaced here.

[0] https://youtu.be/qUAmTYHEyM8?t=1201

tzs2• 10 years ago

Hopefully Google does not do what IBM did and stop playing after they win.

IBM lost in 1996, 2-4, and then won in 1997, 3.5-2.5. If they had played a third match with Kasparov, especially a longer match, it is not at all clear that they would have won.

Kasparov asked for a third match of 10 games, to be played over 20 days, but IBM would not give it to him.

qznc• 10 years ago

What is the point? What a few years and you can get an equivalent AI on your home computer. Play as many games as you want.

Of course, you can change rules as much as you want and generate new marketing events. Personally, I would like a match where the AI is only allowed the energy a human uses. I guess, AlphaGo would lose with only 2000 kcal (2.3 kWh). Not sure about chess.

dwaltrip2• 10 years ago

AlphaGo won solidly by all accounts. This is an incredible moment. We are now in the post-humanity go era.

The one solace was that Lee Sedol got his ko =) however, AlphaGo was up to the task and handled it well.

mikhail-g-kan• 10 years ago

Interestingly, I feel proud of AI, despite humans lost. It's the progress toward our end as classic human species

jamornh• 10 years ago

Based on all the commentaries, it seems that Lee Sedol was really not ahead during the game at any point during the game... and I think everybody has their answer regarding whether AlphaGo can perform in a Ko fight. That's a yes.

awwducks8• 10 years ago

Perhaps the last big question was whether AlphaGo could play ko positions. AlphaGo played quite well in that ko fight and furthermore, even played away from the ko fight allowing Lee Sedol to play twice in the area.

I definitely did not expect that.

Major credit to Lee Sedol for toughing that out and playing as long as he did. It was dramatic to watch as he played a bunch of his moves with only 1 or 2 seconds left on the clock.

hasenj5• 10 years ago

as he played a bunch of his moves with only 1 or 2 seconds left on the clock.

That's on purpose to make full use of time and think about the next move

awwducks8• 10 years ago

As a spectator, I was on the edge of my seat :)

As a 3d amateur, I'm really curious about when he resigned. It really seemed like he was playing the position out to go for the win (or perhaps to see how AlphaGo would fare in ko). It didn't look like he was searching for a place to resign.

partycoder8• 10 years ago

He was just making effective use of his byo-yomi. Pro players are used to that.

massemphasis• 10 years ago

Well, we don't actually know AlphaGo's true strength in Ko matches because the Ko setup wasn't that complex. There wasn't a lot of trade points available.

A possible explanation is:

During self reinforcement learning, AlphaGo learned to minimize Ko potential by maximizing its probability of winning through diminished available Ko moves.

It would be interesting to see how AlphaGo would be able to capitalize on a game that emphasized Ko play, but that would take more time with AlphaGo to emphasize that kind of play.

edit: I'm not sure why, but I think Lee Sedol is partly holding back, or not playing at his maximum ability. It feels like these games are more along the lines of query games.

I look forward to the next two games because I'm 100% certain Lee Sedol is going to query the AI with some new queries.

tel4• 10 years ago

I think from a technological perspective there's very little question that AlphaGo could play ko. I would have imagined that AlphaGo would be better at ko than most human players since it's a question of balancing risk across the entire board. Human players might be more likely to be exhausted and choose suboptimally by the calculation deciding between different stakes on the board, but MCTS will correctly optimize for the long term potential of each major branch in the game tree.

So I'd be very surprised if that turns out to be the trick. Things that are hard for human players are not at all necessarily AlphaGo's weaknesses.

edanm• 10 years ago

Don't have the link offhand, but I read on Reddit that Lee Seedol and a few other Go professionals pulled an all-nighter coming up with ways to beat AlphaGo, and one of their guesses was that AlphaGo would be bad at Ko's. I think the reasoning was because that hadn't happened in the games, so they assumed it was avoiding them.

TheRealPomax• 10 years ago

When you have a minute, even if you already know your move, you take the rest of the time to read the plays. You don't play your stone until there's only 1 or 2 seconds left, unless the play is so trivial that you need to see your opponent's play in order to continue reading. Every time Sedol played with 1 or 2 seconds left, his rush was only to get the stone on the board, not because he suddenly knew what to play.

hasenj5• 10 years ago

Seems to be playing at super-human levels.

I'm no where near a strong player but it seems like AlphaGo is far ahead of Lee Sedol.

mathgenius• 10 years ago

It seems like in all three games, AlphaGo plays a thick move that looks a bit baffling to the commentators, but then miraculously those moves become hugely profitable somewhat later on in the game.

tome• 10 years ago

Is it possible that all of AlphaGo's strength is in these baffling moves? If humans played enough games against AlphaGo and discovered how to counter the baffling moves, is it plausible that AlphaGo's strength would be lost?

hyperion2010• 10 years ago

I really want to see how a team of humans would do against alpha-go with a 3 or 4 hour time limit.

_snydly• 10 years ago

Yeah, if AlphaGO wins all 5, I'd next like to see some Blitz Go (if that's a thing).

daveguy8• 10 years ago

Blitz go would be even more heavily favored toward AlphaGo. AG didn't spend more than 10 seconds or so per move until the meat of the midgame. Human pros were losing blitz chess long before they were losing long game chess.

pmontra3• 10 years ago

A team of go players usually is not much stronger than the best player of the team. I won't be afraid of playing against a team of players a couple of stones weaker than me. None of them can come up with moves beyond their level so their strategy won't improve. They could get more accurate at reading and that's worth something.

seanwilson3• 10 years ago

Super interesting to watch this unfold. So what game should AI tackle next? I've heard imperfect information games are harder for AI...would the AlphaGo approach not work well for these?

jonbaer2• 10 years ago

Hassabis discusses some StarCraft here ... http://www.theverge.com/2016/3/10/11192774/demis-hassabis-in... ... good interview.

seanwilson3• 10 years ago

I read that too. It's interesting StarCraft is suggested instead of any classic board games. It sounds like Go is the pinnacle of board games then.

Would StarCraft be the last game AI has to beat?

raarts• 10 years ago

Are we really sure we want to train our top AI in galactical warfare?

abecedarius5• 10 years ago

One perfect-information game that's at least as hard: constructive mathematics. (Proof assistants even give it a sort of videogame-style UI.) I've been wondering about some kind of neural net for ranking the 'moves' coupled with the usual proof search.

seanwilson3• 10 years ago

I don't know much about Go but I'm guessing general proof automation would be many, many orders of magnitudes harder. The branching factor is huge (you can apply any theorem you want to the current goal and go down a bad path) and knowing if you're on the right track to finish a proof isn't obvious.

canndrew2016• 10 years ago

I asked about this in /r/dependent_types yesterday. Apparently there's already a fair bit or research going on in this area. See the links: https://www.reddit.com/r/dependent_types/comments/49z1uc/usi...

vok• 10 years ago

TIS-100

vamur• 10 years ago

If they make a decent AI for Total War I'd be impressed. Or an AI in Civilization that doesn't cheat. But of course requiring a mini-datacenter in hardware and time to think (probably seconds) would make it unrealistic for both.

Huhty• 10 years ago

Full video will be available here shortly:

https://www.youtube.com/playlist?list=PLqYmG7hTraZA7v9Hpbps0...

(It also includes the videos of the first 2 matches)

awwducks8• 10 years ago

I am really curious about the reviews from An Youngil 8p and Myungwan Kim 9p. The commentary by Redmond always tend to leave something to be desired.

jonah2• 10 years ago

Myungwan Kim 9p and Cho Hyeyeon 9p's commentary on the American Go Association YouTube channel:

Game 1: https://www.youtube.com/watch?v=6ZugVil2v4w

Game 2: https://www.youtube.com/watch?v=EitoPhtGWJQ

Game 3: https://www.youtube.com/watch?v=CkyVB4Nm9ac

awwducks8• 10 years ago

Thanks, jonah!

I recall Andrew mentioning that Myungwan will be back to comment games 4 and 5?

Yuioup3• 10 years ago

I really like the moments when Alpha-Go would play a move and the commentators would look stunned and go silent for a 1-2 seconds. "That was an unexpected move", they would say.

starshadowx23• 10 years ago

In game 2 there was a point where Michael Redmond seemed to do a triple take and couldn't believe the move AlphaGo played.

Yuioup3• 10 years ago

Yeah they seem to forget that Alpha-Go is looking deep into the future. I have not read the Nature paper but I assume it's playing out possible moves way into the future.

At some point it figured that the Ko fight at the bottom was already won. Hence that white move at the top which nobody saw coming.

Another interesting moment was when Michael Redmond said "A human would typically not spend too much time thinking on this obvious move". This was the move on the right-hand side somewhere. What this tells me is that human players rush through some moves because they seem obvious but since Alpha-Go is a machine, it does not care about obvious and non-obvious. It's calculating the entire board through to the end and is not interested in "local fights".

dkopi• 10 years ago

One can only hope that in the final battle between the remaining humans and the robots, it won't be a game of Go that decides the fate of humanity.

mzitelli• 10 years ago

Congratulations to AlphaGo team, curious to see if Lee Sedol will be able to defeat it in the next matches.

tim3332• 10 years ago

Kind of a shame the tournament isn't closer.

abecedarius5• 10 years ago

As Eliezer Yudkowsky tweeted before the start, the chances were that, whoever won, they'd win by like 4-1 or 5-0. There was no strong reason to expect an even match.

oneeyedpigeon9• 10 years ago

I think so too. I actually feel really bad for Lee Sedol if he loses 5-0.

wagglycocks2• 10 years ago

The avalanche has already started, it is too late for the pebbles to vote.

partycoder8• 10 years ago

Some professionals labeled some AlphaGo moves as being unoptimal or slow. In reality, Alpha Go doesn't try to maximize its score, only its probability of winning.

AndyNemmity2• 10 years ago

Right, but I think the initial concerns have been proven wrong in some respect. I think AlphaGo can accurately determine when a move in a local area doesn't significantly change the outcome and then just does something else for a different reason. AlphaGo has a higher willingness to go somewhere else, and I think that's the algorithm finding another path that is more helpful than the current one for 1 move.

Humans have a tendancy to want to win the battle, or to get too focus in a local area. I think that's a way AlphaGo is coming up with an extra move here or there which is making a difference in the fight later.

PeCaN• 10 years ago

From watching it I'm almost inclined to say it maximizes its chances of not losing over necessarily winning.

AndyNemmity2• 10 years ago

The developers have said that. They also said it doesn't care if it wins by 20 or by 1. It just maximizes winning at all.

spicyj• 10 years ago

Sorry, but what's the difference?

tzs2• 10 years ago

They are playing with a 7.5 point komi [1], and so a game cannot end in a tie. Doesn't that mean that there is no distinction between winning and not losing?

[1] komi is a number of points that is added to white's score to compensate for the disadvantage of moving second. When a non-integer komi is used, such as in this match, you cannot have a tie because scoring on the board is always integral.

zhaphod• 10 years ago

Your comment reminds me of the Star Trek TNG episode Peak Performance. Data wins the rematch by playing to stalemate.

https://en.wikipedia.org/wiki/Peak_Performance_(Star_Trek:_T...

starshadowx23• 10 years ago

I'm very interested to see what the Google DeepMind team applies themselves to in the future.

pmyjavec3• 10 years ago

That's going to be getting to know even more about you and the probability of you clicking on an advertisement, stakeholder, shareholder profits!

starshadowx23• 10 years ago

I don't understand your comment.

pushrax3• 10 years ago

It's important to remember that this is an accomplishment of humanity, not a defeat. By constructing this AI, we are simply creating another tool for advancing our state of being.

(or something like that)

partycoder8• 10 years ago

It is not a tool if you can't control it.

Politicians fool countries delivering empty promises about better health, education and security. A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world.

https://en.wikipedia.org/wiki/AI_box

landryraccoon2• 10 years ago

Well, it would have to have a motivation to do so. Evolution has put complex motivations into human beings for billions of years, self preservation being chief among them. Even if we put motivation for self preservation into an AI, we might not do as well as nature did, leaving the AI open to self destruction or shutdown by humans - simply because the AI has no motivation not to allow humans to turn it off. Human designers would do well to ensure that no super intelligent AI has any motivation for self preservation.

Basically, why would an AI want to dominate the world? Humans would have to both very stupidly give the AI values that encourage it to dominate the world and very luckily (or unluckily) give it values that actually converge to a horrible outcome against human intentions by random chance (since the AI designers certainly won't be tuning the value set for that outcome).

215• 10 years ago

Do you feel guilty when you break your promise against your cat? Do you even think for a nanosecond if it's ethical to lie to it?

Of course, a cat is not conscious. But compared to an AI, we might also be considered pretty low consciousness beings, or at least beings in front of which you don't justify yourself.

kotach3• 10 years ago

AlphaGo is certainly controllable.

FLUX-YOU2• 10 years ago

A supraintelligent AI could promise making humans rich, healthy and powerful, to then break its promise and dominate the world.

Or it could devote its entire power to making human lives the best and most comfortable they can be because humanity is some super-precious resource in the universe and it feels it's unimportant because it's just a bunch of silicon and electrons.

Supraintelligent AI being evil is FUD imho because we can't reason about supraintelligent AI.

alextgordon• 10 years ago

What is our purpose if computers can do everything better than us?

It feels like computers have taken one aspect of humanness: logic. Computers could do arithmetic, do algebra, play chess, and now they can play go.

It hurts because logic is usually thought to be one of the highest of human characteristics. Yes computers might never be able to replicate emotion, but even dogs have that.

There's still some aspects we have left to call our own. Computers perform poorly at language-based tasks. They can't write books, write math papers, compose symphonies. I hope it stays that way.

oneeyedpigeon9• 10 years ago

Do you feel the same way about your human children? Would you rather they were 'better' than you or 'worse'?

pokolovsky• 10 years ago

What is the purpose and meaning of anything else? We human being is just a node in the evolution of the whole universe.

atemerev3• 10 years ago

No it won't. Natural languages are the next focal point of AI research. Expect big changes in the next few years.

samizdatum• 10 years ago

You're implying that if something can be outdone it doesn't have a purpose, which seems to rule out purposes for pretty much everything.

I'm sure there's always someone that can write books or maths papers or symphonies better than you. I don't think this robs you of purpose, unless your purpose is to be the absolute best at something.

Anyway, I find it curious that you would say logic is a quintessentially human trait, because humans are naturally quite bad at logic.

nathan_f773• 10 years ago

First we would have to figure out our purpose even if computers couldn't do everything better than us. I don't think many people have answered this question. I believe the answer is something to do with love, reproduction, creation, and happiness.

nathan_f773• 10 years ago

Computers will be able to compose symphonies very soon. If DeepMind started working on this problem, I am sure that they would succeed. At least, we would have some innovative mashups of Beethoven, Mozart and Tchaikovsky. But training a powerful AI on a massive dataset of all popular and classical music should produce some extraordinary results. Especially if the dataset was given as MIDI with separate instrument tracks, so that an AI could learn how to write parts for different instruments, and how a song should be balanced. I actually think we are at the point where we have more than enough data to distill the essence of "good music", and generate an endless supply of great songs.

People have been doing his for decades, but as far as I'm aware, no-one has tried it with thousands of distributed servers and millions of songs.

bad_user• 10 years ago

Reading this thread, I believe there's one aspect not discussed: in a battle between man and machine, it's debatable who wins and depends on the domain, but a man-machine combination always wins over both.

On emotions, that's a characteristic of life. With the consciousness we possess, without emotions we would quickly realize that life isn't worth living. I doubt that a "true AI", one with consciousness, will want to live without emotions. And about dogs, we haven't built anything as sophisticated yet ;-)

On AlphaGo, personally I'm not impressed. It's still raw search over the space of all possible moves, combined with neural networks and these techniques do not have the potential to yield human-level intelligence.

On logic, we have enough as to be able to build AlphaGo (also aided by computers and software that we've built, in a man-machine combination, get it?). Can a computer do anything resembling that yet? Of course not, because for now computers are just glorified automatons.

mirimir3• 10 years ago

Why assume that humans and computers won't merge?

mirimir3• 10 years ago

There is no "purpose".

There is only selection.

Meat is just a phase.

fma4• 10 years ago

Our purpose is to enjoy life... Strive to be the best Go play you can be and enjoy the process.

Don't worry about the machine. Even in Star Trek TNG, Data can outperform everyone in every task, but was never truly happy!

JabavuAdams2• 10 years ago

What is our purpose if computers can do everything better than us?

Use the computers to engineer ourselves to "superhuman" capabilities.

tdsamardzhiev• 10 years ago

I hope it won't stay that way :) If we can create a being smarter than ourselves, we should by all means do so.

Shorel• 10 years ago

Future research will be along these lines.

zardo• 10 years ago

What is our purpose if computers can do everything better than us?

You think we have one now?

baq• 10 years ago

and what is our purpose if there are no computers at all? you can keep believing whatever you believe, computers being good at things change nothing.

evanb• 10 years ago

Will robots inherit the earth? Yes, but they will be our children. -- Marvin Minsky http://web.media.mit.edu/~minsky/papers/sciam.inherit.html

"Naches" from our Machines https://www.edge.org/response-detail/26117

Naches is a Yiddish term that means joy and pride, and it's often used in the context of vicarious pride, taken from others' accomplishments. You have naches, or as is said in Yiddish, you shep naches, when your children graduate college or get married, or any other instance of vicarious pride. These aren't your own accomplishments, but you can still have a great deal of pride and joy in them.
And the same thing is true with our machines. We might not understand their thoughts or discoveries or technological advances. But they are our machines and we can have naches from them.

ap222132• 10 years ago

I hate to be cynical, but I'm sure many of my ancestors were also told to believe similar things. I know for certain that my grandparents and great grandparents believed that technology would create such progress that people of my generation would not have to work, and all would have leisure time.

AI is more likely to evolve into a tool to be used by the few to control the many.

fma4• 10 years ago

Agreed. Even with so much automation and increase in productivity, we (at least In the US) are working just as much, if not more.

orangecat2• 10 years ago

We do in fact have more leisure time: https://www.stlouisfed.org/publications/regional-economist/j.... And that doesn't count reading Facebook at work.

ljk• 10 years ago

Sounds a little like what the villains always say in the movies....

partycoder8• 10 years ago

I don't think Ke Jie would win against Alpha Go either.

pmontra3• 10 years ago

I agree. Lee lost a title 3-2 to Ke Je recently but he lost very convincingly against AlphaGo. Ke Je is not that stronger than Lee. He's going to lose too but he's 18 so I understand that he's feeling he can do anything.

My feeling is that AlphaGo could be a couple of stones stronger than the best pros. Considering that dans are capped at 9 but there are 9 dans that always win against some other 9 dans, the best pros could be 10 dan on their scale. That would also be 10 dan on the amateur scale (European, Americans are usually one stone gentler and Japanese much more than that). AlphaGo could be 12d amateur or, don't know, 14-15 dan pro? (pro levels are on a finer scale)

oneeyedpigeon9• 10 years ago

Do you know what the current limit is on a handicap that a professional player can overcome? Is there a known theoretical limit? That's a possible 'endgame' for go ai.

jug• 10 years ago

No, I'm skeptical. And most importantly, due to how AlphaGo seems to be raising the bar just enough to lead confidently, I think Ke Je is in no position to judge how he would fare in advance. I mean, we have absolutely no idea whatsoever as for AlphaGo's true potential.

eemax• 10 years ago

What happens if you give the human player a handicap? I wonder if the games are really as close as the commentators say, or if it's just a quirk of the MCST algorithm.

ConbrastanJorji• 10 years ago

The great go champion Otake Hideo famously said that if he were to play go against God Himself, he would take only a three stone handicap, and if his life depended on it, he would take four. Alphago's not perfect, but it would still be very interesting to watch such a handicapped game.

pgodzin• 10 years ago

Can it even play a handicapped game? I'm assuming the handicap means the other play can begin with several stones on the board. AlphaGo would have no training on such a configuration and would likely not know how to approach it.

jonah2• 10 years ago

Cho Hyeyeon 9p's commentary on the American Go Association YouTube channel: https://www.youtube.com/watch?v=CkyVB4Nm9ac

awwducks8• 10 years ago

I guess the next question on my mind is how AlphaGo might fare in a blitz game.

yeukhon2• 10 years ago

I wonder what if you put the top 10 players in a room, team up and play with Alpha-Go. They are allowed to make one move within a 1-hour period and they can only play up to 8 hours a day. I wonder what the outcome would be.

Anyway, I think AlphaGo is a great training companion. I think Lee felt he's learning.

Finally, I also feel that while experience is crucial, the older generation would flush out by the younger generation every decade. I wonder if age really play a role in championship - not that AlphaGo isn't considered a 1000 years old "human" given it has played thousands of games already.

flyingbutter• 10 years ago

The Chinese 9 Dan player Ke Jie basically said the game is lost after around 40 mins or so. He still thinks that he has a 60% chance of winning against AlphaGo (down from 100% on day one). But I doubt Google will bother to go to China and challenge him.

Radim4• 10 years ago

Challenging Ke Jie is way too small a goal for DeepMind at this point.

I wonder if even the idea from the AGA stream today, to get all the best pros in the world together and challenge AlphaGo as a team, is enough.

Perhaps releasing the core AlphaGo as open source (to the extent it's not dependent on internal Google machinery), or at least publishing its trained model, may be the next step. Let people "challenge themselves" however they want.

EDIT: Also, Lee Sedol had his time in the sun, but commiserations to Ke Jie. He's just 19, already number #1 in the world, his whole career in front of him... and this happens.

nkurz• 10 years ago

I wonder if even the idea from the AGA stream today, to get all the best pros in the world together and challenge AlphaGo as a team, is enough.

Has this been tried? That is, have Players 2-9 (or some subset) ever competed as a group against a dominant Player 1? Unless it's been tested, I wouldn't take it for granted that a group would beat an individual.

bluecalm• 10 years ago

I really hope they bother with one more match at least. It was a pity human vs computer idea basically died in chess after very unconvincing win by Deep Blue (even though it lost 2 previous matches). Humans had at least a few years more of good resistance back then.

I can think of a worthy goal for AlphaGo: make a program which can play better than top pros which runs on a macbook pro.

pedrosorio• 10 years ago

Regarding Ke Jie's future, Magnus Carlsen doesn't seem to be doing too bad, and DeepBlue happened almost 20 years ago.

rhaps0dy• 10 years ago

and this happens

Chess competitions are still going strong.

Also, human runners do not compete against cars.

partycoder8• 10 years ago

AlphaGo has also improved very quickly. Without doubt, the AlphaGo seen playing against Fan Hui would have lost against Lee Sedol. But in a couple of months its playing level raised significantly.

Lee Sedol said he could beat AlphaGo, based on the Fan Hui games. Ke Jie said he could beat AlphaGo, based on the Lee Sedol games.

Ke Jie belongs to a similar category than Lee Sedol, and we could see how Lee Sedol was completely dominated by AlphaGo, 3-0 so far. It is not unreasonable to say AlphaGo will most likely beat Ke Jie, and even if that doesn't happen the first time, AlphaGo can be improved by adding more infrastructure and training time.

nsajko• 10 years ago

That is obvious, and Ke Jie also said so.

tunichtgut5• 10 years ago

He said with the same conditions, his chances are 40:60 in favor of him winning.

tel4• 10 years ago

While AlphaGo has been improving, it's also a little hard to see whether AG playing Fan Hui wasn't already strong enough to defeat LSD. MCTS picks the move most probable to win and if AG gets an early advantage it'll play slack for the rest of the game taking moves which never reduce its win rate even if they look boring and weak.

bronz• 10 years ago

Once again, I am so glad that I caught this on the live-stream because it will be in the history books. The implications of these games are absolutely tremendous. Consider GO: it is a game of sophisticated intuition. We have arguably created something that beats the human brain in its own arena, although the brain and AlphaGO do not use the same underlying mechanisms. And this is the supervised model. Once unsupervised learning begins to blossom we will witness something that is as significant as the emergence of life itself.

codecamper• 10 years ago

Yes. We will witness the birth of Marvin the Android.

danmaz742• 10 years ago

Consider GO: it is a game of sophisticated intuition

It's still a game that can be described in terms of clear state-machine rules. The real challenge for AI is making sense and acting in the real world, which can't be described in such way. I consider advances in self-driving cars much more interesting in that sense - even if, even there, there are at least some rule-based constraints that can be applied to simplify the representation of the "world state".

oneeyedpigeon9• 10 years ago

But AlphaGo isn't an enormous state machine ... is it?

chubot2• 10 years ago

Yeah, I think it's accurate to say that driving a car is harder in some sense than playing Go, based on the fact that the AI for Go came first. A lot of people have been working on self-driving cars for a long time now, and it's very monetizable, unlike Go.

Radim4• 10 years ago

In a recent interview [1], Hassabis (DeepMind founder) said they'd try training AlphaGo from scratch next, so it learns from first principles. Without the bootstrapping step of "learn from a database of human games", which introduce human prejudice.

As a Go player, I'm really excited to see what kind of play will come from that!

[1] http://www.theverge.com/2016/3/10/11192774/demis-hassabis-in...

awwducks8• 10 years ago

That would be amazing if it could achieve the same levels (or higher) without the bootstrapping.

The niggling thought in my mind was that AlphaGo's strength is built on human strength.

radicalbyte• 10 years ago

Human strength is also "built on human strength" so I don't see the problem? :)

3minus1• 10 years ago

I'm pretty sure that the reinforcement learning algorithm they are using is guaranteed to converge. It just takes a very long time to train, and using human games probably sped it up.

1024core• 10 years ago

What if the "betaGo" played just AlphaGo, and learned from its games?

BTW: even humans don't just randomly pick up the game. They have teachers, who teach them the tricks of the trade and monitor their games.

inopinatus• 10 years ago

I suspect you will only be satisfied when AIs play each other at an incomprehensible game of their own devising.

kurlberg• 10 years ago

A case of life imitating AI koans:

Uncarved block

In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied. "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play", Sussman said. Minsky then shut his eyes. "Why do you close your eyes?" Sussman asked his teacher. "So that the room will be empty." At that moment, Sussman was enlightened.

(It seems based on a true story https://en.wikipedia.org/wiki/Hacker_koan )

argonaut3• 10 years ago

The actual quote: "Sussman told Minsky that he was using a certain randomizing technique in his program because he didn't want the machine to have any preconceived notions" makes no indication it was a neural net ;)

jmathes• 10 years ago

It sounds like the point of this story is to illustrate by analogy that starting from first principles is sometimes a silly way to approach a problem, and by extrapolation that it's a silly way to make an AI that plays Go well.

Making an AI that plays Go well is not (and has never been) the real goal. They're trying to learn how to build a AI that can solve any problem.

wagglycocks2• 10 years ago

I feel like an ant in the presence of giants.

kotach3• 10 years ago

I believe the whole point of pretraining on reference policies, which a collection of "optimally" played human games is, is just avoidance of bad local optimum.

It can be a case that training and learning on just a learned policy is going to get you stuck in a local optimum that is of worse quality than the one with pretraining.

If they stored all of the AI played games their reference policy (the data) would be of extreme value. You could train a recurrent neural network, without any reinforcement learning, that you could probably run on a smartphone and beat all of the players. You wouldn't need a monte carlo search too.

There are algorithms [1] that have mathematical guarantees of achieving local optimality from reference policies that might not be optimal, and can even work better than the reference policy (experimentally) - assuming that the reference policy isn't optimal. The RNN trained with LOLS would make jointly local decisions over the whole game and each decision would guarantee that a minimization of future regret is being done. Local optimality mentioned here isn't finding a locally optimal model that approximates the strong reference policy, it means that it will find the locally optimal decisions (which piece to put where) without the need for search.

The problem is that for these algorithms you have to have a closely good reference policy, and given a small amount of human played Go games, reinforcement learning was the main algorithm instead, it allowed them to construct a huge number of meaningful games, from which their system learned, which allowed them to construct a huge number of more meaningful games, etc.

But, now when they have games that have a pretty good (AlphaGo is definitely playing on a superhuman level) reference policy, they can train the model based on that reference policy and they wouldn't need a search part of the algorithm at all.

The model would try to approximate the reference policy and would definitely be worse than AlphaGo real-search based policy, but it wouldn't be significantly worse (mathematical guarantee). The model is trained starting from a good player, and it tries to approximate the good player, on the other hand, reinforcement learning starts from an idiot player, and tries to become a good player, reinforcement learning is thus much much harder.

[1] http://www.umiacs.umd.edu/~hal/docs/daume15lols.pdf

wnkrshm• 10 years ago

While he may not be number one in the Go rankings afaik, Lee Sedol will be the name in the history books: Deep Blue against Garry Kasparov, AlphaGo against Lee Sedol. Lots of respect to Sedol for toughing it out.

theroof• 10 years ago

Is anyone also asking themselves when they'll be able to play against this level of AI on their mobile phone? Or formulated differently: when will an "AlphaGo" (or equivalent) app appear in the play/app store?

In 2 years? In 1 year? In 3 months?

pixl975• 10 years ago

Well, we can't answer that question at this point. We do know that the system is more powerful than a human, so the question becomes "How much hardware can we take away and still be better than the best". That sets the hardware floor we have to reach.

The current answer is, still a long way off. With massive forecasts in GPU power over the next few years it may be reasonable for people to have desktops in their home with this kind of power possibly within a decade. I won't even give a timeline for phones because of the power requirements.

tunichtgut5• 10 years ago

I think this is a good guess. 280 GPUs in a single Desktop. It will take at least 10 years.

Perhaps longer given the restrictions (and huge problems) in further down scaling of the known silicon technology. GPUs are now at 28nm and the newest Intel CPUs on 14nm.

agildehaus• 10 years ago

A version of AlphaGo that beat Fan Hui ran on a single machine, but a very beefy one (48 CPUs and 8 GPUs).

Unless they can optimize it quite a bit, we won't be seeing this on a mobile phone anytime soon. Perhaps you'll be able to pay a good hourly rate for the cloud version.

iopq4• 10 years ago

A mobile phone version will still trounce the current AIs like CrazyStone considering the single machine version is competitive with the cluster (can take some games off of it)

Eliezer2• 10 years ago

My (long) commentary here:

https://www.facebook.com/yudkowsky/posts/10154018209759228

Sample:

At this point it seems likely that Sedol is actually far outclassed by a superhuman player. The suspicion is that since AlphaGo plays purely for probability of long-term victory rather than playing for points, the fight against Sedol generates boards that can falsely appear to a human to be balanced even as Sedol's probability of victory diminishes. The 8p and 9p pros who analyzed games 1 and 2 and thought the flow of a seemingly Sedol-favoring game 'eventually' shifted to AlphaGo later, may simply have failed to read the board's true state. The reality may be a slow, steady diminishment of Sedol's win probability as the game goes on and Sedol makes subtly imperfect moves that humans think result in even-looking boards...

The case of AlphaGo is a helpful concrete illustration of these concepts [from AI alignment theory]...

Edge instantiation. Extremely optimized strategies often look to us like 'weird' edges of the possibility space, and may throw away what we think of as 'typical' features of a solution. In many different kinds of optimization problem, the maximizing solution will lie at a vertex of the possibility space (a corner, an edge-case). In the case of AlphaGo, an extremely optimized strategy seems to have thrown away the 'typical' production of a visible point lead that characterizes human play...

taneq• 10 years ago

This is reminding me more and more of the central theme of the book Echopraxia, which was that when dealing with superhuman intelligences, you are by definition not smart enough to even know whether you're winning.

studentrob5• 10 years ago

Hmm maybe but aren't humans capable of learning too?

One could make the same statement of two humans of different ranks playing each other. In some cases, the more highly ranked player might be the only one who knows who's winning.

This doesn't mean that the lesser ranked player can't improve over time. It just means that at that moment, that player is inexperienced or less skilled, which we already knew.

jules2• 10 years ago

When I play chess against a computer I can see very easily that I'm not winning.

monk_e_boy• 10 years ago

Have you read Existence by David Brin, it has a lot of great AI stuff in it.

studentrob5• 10 years ago

I wonder if he could identify those long-term-thinking moves by playing a version of AlphaGo on a smaller board, then gradually increasing the size of the board over successive games once he is able to consistently win or tie at a given board size.

I also wonder if when Alpha Go plays itself, does it always win when it is black or white, does it always tie, or is it a mix?

So much to learn from this.

oneeyedpigeon9• 10 years ago

I was thinking the opposite. AI has already been beating humans on smaller boards, so maybe the only chance we have is to start playing go on bigger and bigger boards.

nathan_f773• 10 years ago

There is typically a point advantage of around 7 points when you get the chance to play the first move. In professional matches, white will be given a 6.5 point bonus to compensate for this. So yes I think it's fair to assume that black usually wins. Actually, AlphaGo could probably be used to calculate the perfect number of points, because it has changed over the years. In China they have a 7.5 point bonus for white.

RikNieu• 10 years ago

Isn't this what the machines learned in the Matrix?

alexlikeits1999• 10 years ago

Regarding the expectation that it would have gone 5-0 or 4-1 to either one, you'd have to have equally predicted a similarly one-sided result in the Deep Blue v Kasparov match. Rate of improvement might end up being completely domain-dependent, and hard to predict a-priori how it will turn out.

YeGoblynQueenne13• 10 years ago

> At this point it seems likely that Sedol is actually far outclassed by a superhuman player.

I still don't agree that this is the case, and I don't care what a thousand Google-hyped press releases say, beating the best human player in anything is not "superhuman" and "superhuman" performance has not been achieved by anything yet. [1]

Why do I think so? Two reasons.

One, because you can be entirely human and still beat all other humans, without fail for a very long time. Long "winning streaks" in professional sports are very well documented. For instance Rocky Marciano went entirely undefeated in his whole heavyweight boxing career. In chess, Mikhail Tal went undefeated for 95 games. And so on, so forth.

Of course most humans' winning streaks end eventually. That's because our performance degrades over time. When a computer wins against the best player, it keeps on winning, and in fact it actually gets even better over time.

Still, and this is number two: in most instances where a computer does better than a human, we don't claim superhuman performance. Automatic calculators, going back to mechanical calculators, have been better than humans at arithmetic for a very, very long time. I would wager that nobody discusses pocket calculators as exhibiting "superhuman" performance. You only hear this sort of claim when it comes to Deep Blue, AlphaGo or Watson.

So maybe we need a better definition of what it means to be "superhuman" that covers both pocket calculators and AlphaGo. Without one, I don't accept that the performance of AlphaGo can be said to be superhuman, unless pocket calculators' performance is also celebrated as superhuman.

_______________________

[1] I'm perfectly willing to go even further than that and say that we can't make machines that have superhuman intelligence and that even if we did, we wouldn't be able to recognise them (this last bit is similar to what the GP says).

m0g• 10 years ago

Your pocket calculator is superhuman at arithmetic. Alphago is superhuman at go. It's not a big deal.

empath753• 10 years ago

There used to be jobs for thousands of people to do what pocket calculators do. Those jobs have been gone for decades. Humans entirely replaced by machines. So, yes, calculators are superhuman.

eggy• 10 years ago

I agree somewhat, but then what is your gauge for superhuman if not some type of competition? How do you evaluate it? On another note, Rocky Marciano had the mafia behind him. Harry Haft fought him and was knocked out, and later claimed the mafia told him he had to throw the fight [1]. There's also a good graphic novel about this. Maybe the AI's goons have threatened Sedol or kin ;)

   [1] https://en.wikipedia.org/wiki/Harry_Haft

pacala• 10 years ago

The suspicion is that since AlphaGo plays purely for probability of long-term victory rather than playing for points [snip]

I wonder how many handicap stones do Lee Sedol, or Ke Jie for that matter, need to have a shot at winning even one game.

ceejayoz• 10 years ago

Now that's a really interesting question. Give AlphaGo progressively worse losing positions and see if it can rescue what the human masters would consider a disaster.

omphalos• 10 years ago

the stark degree to which "AlphaGo output that move because the board will later be in a winning state" sometimes doesn't correlate with conventional Go goals like taking territory or being up on points

Minor quibbles: 1) taking territory or being up on points are the same thing, and 2) in go there are many other things to optimize for besides territory/points - influence, shape, efficiency, and sente being examples of this.

drited• 10 years ago

How do we know that it's not just that the AI dominates the later stages where the number of permutations has reduced to the point where AI can calculate optimal moves? In other words, how do we know that the computer has an edge on the early stages and isn't just breaking even until the later stages with less permutations arrive, at which point it dominates because there are still too many for humans to assess?

hasenj5• 10 years ago

At least in the case of AlphaGo it was pretty clear that it was dominating the board early on.

bratfarrar• 10 years ago

The implication is that humans have been trapped in a local maxima in terms of strategy and board reading. The question then becomes whether we can use this to bootstrap ourselves.

Praxilla• 10 years ago

I agree with you as this makes perfect sense. I'm new to the game Go but have been riveted by these games and the commentary.

Here's where I think you're right. It seems from the commentary during the matches that the pro player thought the game to be close...on every match.

Then the tides turned...or so we thought...and the game always swayed in AlphaGo's favor.

I like how they were saying that the computer changed up the game style as well.

All of this is very interesting, and I see that they have games 4 and 5 listed however I wonder if they are going to go thru with them and play them as scheduled.

Maybe to see if AlphaGo can sweep the series? Or show it wasn't a fluke of any kind?

I hope they play the last two...just to see how AlphaGo plays them. :D

daveguy8• 10 years ago

They will play all 5 games in the match. The result can end up 3-2 4-1 or 5-0. Based on the past 3 which ended as sound defeats for Lee with different play styles and approaches (aggressive, calm and a mix typical of his usual game) I expect the final result will be 5-0 for AlphaGo.

contraban• 10 years ago

Or its just playing along until the state is complex enough, when it outperforms Seoul simply in terms of memory. I'd call that mid-term, my offer as a compromise ;)

dfan• 10 years ago

For all we know from what we've seen, AlphaGo could win even if Sedol were allowed a one-stone handicap.

What do you mean by a one-stone handicap? Just no komi?

In the case of AlphaGo, an extremely optimized strategy seems to have thrown away the 'typical' production of a visible point lead that characterizes human play. Maximizing win-probability in Go, at this level of play against a human 9p, is not strongly correlated with what a human can see as visible extra territory - so that gets thrown out even though it was previously associated with 'trying to win' in human play.

Pros have been aware for centuries that "visible extra territory" is a poor indicator of win probability. They use the term "thickness" (in English) to denote the positive potential resulting from a strong, safe, influential position, independent of current territory. Quite often a pro game will be a battle of territory vs thickness.

ionforce2• 10 years ago

AI alignment theory

What is this?

sbierwagen2• 10 years ago

The problem of making sure an AI does what you want it to.

http://lesswrong.com/lw/n4l/safety_engineering_target_select...

YeGoblynQueenne13• 10 years ago

> But Go is rich enough to demonstrate strong cognitive uncontainability on a small scale. In a rich and complicated domain whose rules aren't fully known, we should expect even more magic from superhuman reasoning - solutions that are better than the best solution we could imagine, operating by causal pathways we wouldn't be able to foresee even if we were told the AI's exact actions.

Hang on. Where are we going to find this magic agent that can do "even more" in "a rich and complicated domain whose rules aren't fully known" (the real world, as opposed to a Go board) than in a game of Go? What are we going to train such a learner with, if we ourselves don't fully know the rules of the domain, as you point out?

Even if a learner somehow magically found a superhuman path to perfect reasoning which is unavailable to us, entirely by accident and purely on its own, why would we select it from other trained learners to keep and foster further, if we think it's actually pretty dumb, rather than magically smart?

You're saying at some point that a fantastic paperclip maximiser might achieve superhuman intelligence and then lie in waiting, poised to turn us all into paperclips only when it knew it was safe to make its move, basically. But, how is it going to become that smart in the first place? It has to be smart enough to know that it must bide its time, but dumb enough that its time hasn't come yet. Sounds like a bit of an impossible double-bind there.

Just because sufficiently advanced AI may look like magic at first, it doesn't mean we should start all believing in magic because it may just be sufficiently advanced AI in disguise.

hacker423• 10 years ago

What are we going to train such a learner with, if we ourselves don't fully know the rules of the domain, as you point out?

This does not make any sense if you assume general intelligence. A physicist who makes a discovery also was not trained to know about this new law or rule beforehand (how could he?).

YeGoblynQueenne13• 10 years ago

> AlphaGo's core is built around a similar machine learning technology to Deepmind's Atari-playing system - the single, untweaked program that was able to learn superhuman play on dozens of different Atari games just by looking at the pixels, without specialization for each particular game.

Woa there. The Deepmind Atari-playing AI was too specialised for each particular game. It had a reward function that translated the score for it. It couldn't learn the importance of the game score on its own, just from "looking at the pixels" and it couldn't learn the significance of the score display and how it changed as a result of its actions on its own. All this had to be hand-coded. And if I've seen a hint that this means it couldn't really-really generalise to other games, like Deepmind claimed, then that's this bit that you report yourself:

> Deepmind ... did reuse the widely known core insight of Monte Carlo Tree Search.

That is no mere detail. That is the crux of the matter, right there. Deepmind used their architecture to improve MCTS far enough that it could beat Lee Se-Dol. They didn't just add MCTS to their already general-game playing system.

Because the original Atari-playing AI was completely useless for playing Go, a game that doesn't have a score and looks nothing like an Atari game. So it wasn't very general at all, despite Google's and DeepMind's claims to the contrary.

moyix2• 10 years ago

I think you're wrong here. From the Nature paper describing AlphaGo:

"We also tested against the strongest open-source Go program, Pachi, a sophisticated Monte Carlo search program, ranked at 2 amateur dan on KGS, that executes 100,000 simulations per move. Using no search at all, the RL policy network won 85% of games against Pachi."

AlphaGo does use MCTS, but it seems that most of its improvements are actually coming from the deep reinforcement learning approach.

joelthelion• 10 years ago

the maximizing solution will lie at a vertex of the possibility space

Keep in mind that as well as AlphaGo plays, it's still extremely far from playing optimal moves.

skybrian• 10 years ago

On the other hand, in many ways AlphaGo plays much like a human professional. The openings and fights were not radically different. The professional commentators did not have a hard time explaining why most moves were good. The structure of the board was understandable. Some moves were "forced" moves that were predictable and many others were good moves that a human would play. The human player isn't efficient enough to win, but attacks and defenses still mostly work as intended.

This suggests that Go space was already very well understood and there aren't radically different play styles we've somehow overlooked. AI is not a magic wand that radically changes how the game works.

However there may be some bias due to AlphaGo having trained on human games and playing against a human. The real proof will happen when they redo the retraining from scratch.

adrianN2• 10 years ago

AlphaGo learned the game from humans by looking at recorded games. It's not suprising that its style mimicks its master's.

rwallace• 10 years ago

The problem with 'flag and footprints' demonstrations is precisely the same as their attraction - they are, in essence (even though that was not the intent), selected to be as misleading as possible about progress in solving the real problems.

We put a man on the moon, which encouraged decades of optimism about the near-future colonization of space... unfortunately it turns out we are nowhere near solving the hard problems associated with being able to live in space.

We spelled out the letters IBM in xenon atoms, five atoms tall, which encouraged optimism about near-future molecular manufacturing... unfortunately it turns out we are nowhere near solving the hard problems associated with making useful things with atomic precision.

We achieve superhuman performance in simple games, which encourages what should be optimism that becomes reflected by the distorting mirror of the zeitgeist of the times into pessimism, about near-future AI... unfortunately it turns out we are nowhere near solving the hard problems associated with making our tools something better than depressingly dumb.

In each case, the flurry of optimism turns as the decades pass into confusion, anger, soul-searching about why the expected follow-on progress has failed to materialize, fading slowly into despair, then into nothing as the optimists die of old age, one by one, and the rate of technological progress becomes slower with every passing decade. For this conversation is necessarily happening at a tech level that allows it, but that provides no guarantee whatsoever that the species that reached tech level X will proceed to tech level X+1. Moore's law has officially expired, and that was the last major area where we were making rapid sustained progress.

It's hard to say we shouldn't do 'flag and footprints' demonstrations; am I really willing to bite the bullet and say we shouldn't have put a man on the moon? I don't know the answer to that. I am, however, convinced that we should remember not to take them too seriously.

nostrademons2• 10 years ago

The "flag and footprints" demonstrations serve essentially as prototypes. They demonstrate that a certain tech level is possible. But as anyone who's built a prototype knows, when the prototype works you are maybe 5% of the way there.

We put a man on the moon in 1969, and we are just now getting privately-funded spaceflight and satellites. We made the first computers in 1945; we got them on every desk in the 1980s. The first steamboat was built in 1783; the transatlantic shipping industry didn't transition to steam until the 1840s & 1850s.

Probably some kid who's watching AlphaGo's game today is the person who, late in life, invents the first strong AI.

ilaksh• 10 years ago

Sorry but more research will reveal massive progress in the fields connected to those ideals. Please research some more, you have missed quite a few developments.

For example, long stays in large space stations, landing craft on Mars, successful isolation sustainability experiments, 14nm chips and understanding of quantum concerns, multi-layer chips, beginnings of optical computing, biological/genetic chemical synthesis, neuromorphic chips, etc.

Deep reinforcement learning is undeniably a major step forward for general intelligence.

You and others with your belief system will still be in denial up to and past the point where your species becomes irrelevant as the superintelligent AIs arrive within a few decades.

bricemo2• 10 years ago

I think this is one of the most fascinating (and disturbing) facts about AI: they seek to solve the problem they are given without being dependent on convention. They are completely alien in this regard, so many of their choices don't make any sense to us. This results in huge possibility but also the huge dangers that people worry about.

dynjo• 10 years ago

How long before AlphaGo also participates in the post-match press conference I wonder...

Jach• 10 years ago

In the first press conference I was half expecting that Google would translate and speak the Korean for the livestream, and not even point it out as a very big deal. If anyone asks, it's just an "oh yeah, we solved that problem ages ago." But more seriously, those translators were very good, they didn't seem troubled at all by the long speeches.

zhouyisu• 10 years ago

Next move: how about beating human at online dating?

femto113• 10 years ago

Pretty sure that's already been done

http://gizmodo.com/how-ashley-madison-hid-its-fembot-con-fro...

gandalfu2• 10 years ago

Optimizing your chances at online dating? Done:

http://www.wired.com/2014/01/how-to-hack-okcupid/

xuesj• 10 years ago

This is a milestone of AI in history. The ability of AlphaGo is amazing and far beyond to human.

tunichtgut5• 10 years ago

I dont know if AlphaGo is "beyond" all human players.

Given AlphaGo has learned all played games from a database, its more like AlphaGo is the essence or the sum of all experience gained from those games played. AlphaGo is better than every player (perhaps), but it has not pushed the game further by itself.

daveguy8• 10 years ago

Not all games came from a database. It was initialized with a database. It has played millions of games against itself to improve beyond that. Next they will skip the bootsrap initialization and use all self play.

ptbello• 10 years ago

Does anyone have insights on how a game between two Alpha-Gos would play out?

vl2• 10 years ago

One would win and another loose. It plays itself all the time to train.

arek_• 10 years ago

I was using machine learning in computer chess some time ago. My commentary: http://arekpaterek.blogspot.com/2016/03/my-thoughts-on-alpha...

bainsfather• 10 years ago

It is interesting how fast this has happened compared to chess.

In 1978 chess IM David Levy won a 6 match series 4.5-1.5 - he was better than the machine, but the machine gave him a good game (the game he lost was when he tried to take it on in a tactical game, where the machine proved stronger). It took until 1996/7 for computers to match and surpass the human world champion.

I'd say the difference was that for chess, the algorithm was known (minimax + alpha-beta search) and it was computing power that was lacking - we had to wait for Moore's law to do its work. For go, the algorithm (MCTS + good neural nets + reinforcement learning) was lacking, but the computing power was already available.

gandalfu2• 10 years ago

Its a matter of language.

Our model of representation of Go fails at expressing the game/strategies of AlphaGo is showing, we are communicating in the board in different languages, no wonder everyone looking the at games is stomped by the machine "moves".

Our brains lack the capacity of implementing such algorithms (understanding such languages), but we can still create them. We might see in the future engine A played against engine B and enjoy the matches.

No one is surprised by a machine doing a better job with integer programming/operational research/numerical solutions etc.

Queribus2• 10 years ago

Strictly speaking, "just" because Alphago finally won, that doesnt mean it was right when claiming being ahead already.

atrudeau• 10 years ago

It would be nice if AlphaGo emitted the estimated probability of it winning every time a move is made. I wonder what this curve looks like. I would imagine mistakes by the human opponent would give nice little jumps in the curve. If the commentary is correct, we would expect very high probability 40-60 minutes into the game. Perhaps something crushing, like 99,9%

Symmetry• 10 years ago

It did, apparently. I wish that information was public.

http://www.nature.com/news/the-go-files-ai-computer-wins-fir...

For me, the key moment came when I saw Hassabis passing his iPhone to other Google executives in our VIP room, some three hours into the game. From their smiles, you knew straight away that they were pretty sure they were winning – although the experts providing the live public commentary on the match that was broadcast to our room weren’t clear on the matter, and remained confused up to the end of the game just before Lee resigned. (I'm told that other high-level commentators did see the writing on the wall, however).

oliebol• 10 years ago

Watching this felt like watching a funeral where the commentary was the eulogy.

eternalban• 10 years ago

My impression is that Sedol was psychologically defeated at 1-0. Computational machines don't crack under pressure - at most they get exhausted.

asdfologist• 10 years ago

Ke Jie's gonna be next.

http://www.shanghaidaily.com/national/AlphaGo-cant-beat-me-s...

"Demis Hassabis, Google DeepMind's CEO, has expressed the willingness to pick Ke as AlphaGo's next target."

asmyers2• 10 years ago

Does anyone know if the AlphaGo team is saving the probability of winning assignments that AlphaGo gave its moves?

It would be fascinating to see how early AlphaGo assigned very high probability of it winning. It would also be interesting to see if there were particular moves which changed this assignment a lot. For instance, are there moves that Lee Sedol made for which the win probability is very different for the AlphaGo moves before and after?

bricemo2• 10 years ago

In the introduction to the third match, the project lead talked about how Alphago evaluated the controversial 37th move in the second match. Apparently the policy network decided that it had a one in 10,000 chance of a human making that move, But the value network overrode this. So it seems like they are recording this kind of data and looking at it in cases where it's interesting.

asmyers2• 10 years ago

Thanks, I'm only just watching match 2 and hadn't seen that. I really hope they publish this data. It would be incredibly interesting to construct a "review of the game by AlphaGo" from this data.

yodsanklai• 10 years ago

All this excitement makes me want to learn a little bit about those algorithms. I don't know anything about neural networks (but I've already implemented a chess game a while ago). Would it be difficult to implement a similar algorithm for a simpler game?

cpeterso• 10 years ago

Does AlphaGo run in the cloud or is it a machine onsite at the match? I wonder how small AlphaGo could be scaled down and still beat Lee Sedol. How competitive would AlphaGo be running on an iPhone? :)

ganwar• 10 years ago

Incredible news. We have all heard all of the positive coverage and how tremendous it is. What I find interesting is that how come nobody is talking about the potential of AlphaGo as a war strategizing AI?

If you provide terrain(elevation etc.) information, AlphaGo can be used to corner opponents into an area surrounded by mountains where AlphaGo is sitting on the mountains. We all know what happens after that.

Don't want to kill the party but I am completely surprised with the lack of chatter in this direction.

Gambit89• 10 years ago

This is because of ethical reasons: one of the conditions that Deepmind made with Google in their deal was for an "embargo on using its technology for military and intelligence applications."

Source: http://www.theguardian.com/technology/2016/feb/16/demis-hass...

jcyw• 10 years ago

We had Godel on the limitation of logic and Turing on the limitation of computation. I think AI will only change the way human calls intelligence. We used to call people who can mentally calculate large numbers genius. Lots of that has to be re-defined.

skarist2• 10 years ago

We are indeed witnessing and living a historic moment. It is difficult not to feel awestruck. Likewise, it is difficult not to feel awestruck at how a wet 1.5 kg clump of carbon-based material (e.g. Lee Sedol brain) can achieve this level of mastery of a board game, that it takes such an insane amount of computing power to beat it. So, finally we do have a measure of the computing power required to play Go at the professional level. And it is immense, or to apply a very crude approximation based on Moore's law, it requires about 4096 times more computing power to play Go at the professional level than it does to play chess. Ok, this approx may be a bit crude :)

But maybe this is all just human prejudice... i.e. what this really goes to show is that in the final analysis all board games we humans have inveted and played are "trival", i.e. they are all just like tic-tac-toe just with a varying degree of complexity.

kotach3• 10 years ago

Chess can be played godlike on a smartphone. Result of years of refining algorithms.

Same could probably be accomplished with Go.

ChuckMcM• 10 years ago

I don't think it is human prejudice, Deepmind can play Go and it cannot drive a car, Lee Sedol's brain can do both. Understanding the nature of cognition will open up new capabilities just as understanding the nature of genetics will open up new capabilities or understanding the nature of the cosmos Etc.

What is most interesting for me is that the nature of solving the problem "how do I win at Go?" is one that has not been, historically, one that computers could solve. Compute the ballistic trajectory of an artillery shell? Easy. Compute a winning strategy on the fly? Impossible. But by creating tools that allow computers to work on those problems we open up the things that can be improved and automated and that has historically improved the experience.

skarist2• 10 years ago

I wholeheartedly agree. I just had a "we feeble humans" moment when I made my comment about our prejudice. And as you point out Sedol's brain can of course do a lot more than just play Go. So, my feeling of feebleness on behalf of humanity was thoroughly unfounded! In fact, it can be argued that our brains are probably too powerful computers for our own good. But who knows, maybe things will conspire and turn us into seals living of some obscure beach on the Galápagos in about a million years. (https://en.wikipedia.org/wiki/Gal%C3%A1pagos_%28novel%29)

kybernetikos2• 10 years ago

Go was the last perfect information game I knew where the best humans outperformed the best computers. Anyone know any others? Are all perfect information games lost at this point? Can we design one to keep us winning?

eli_gottlieb• 10 years ago

Why bother? It's working with radically incomplete and imperfect information where the core of human intelligence actually lies.

kybernetikos2• 10 years ago

Why bother?

Because the process might tell us something interesting and useful about AI and ourselves.

mrob• 10 years ago

I know of one attempt to design such a game: Arimaa. Playable on a chess board, Arimaa has an even bigger game tree than Go. It's a great game, and fun to play, but as an anti-computer design it has recently failed.

There has been an annual humans vs computers challenge match every year since 2004, and in 2015 computers won with David Wu's software "Sharp". Despite the very high branching factor, standard chess AI techniques turned out to be applicable when combined with high quality hand written heuristics for positional evaluation and candidate move generation. The software is described in detail here:

http://icosahedral.net/downloads/djwu2015arimaa_color.pdf

pmyjavec3• 10 years ago

If one allowed AlphaGo to train forever, what would happen? Would it constantly just tie against itself ?

Queribus2• 10 years ago

Was I in a "prophetic mode" yesterday? ;)))

AlphaGo beats Lee Sedol 3-0 [video]

Related Stories