How I Fail – Ian Goodfellow

http://www.veronikach.com/how-i-fail/how-i-fail-ian-goodfellow-phd14-computer-science/

by harias • 8 years ago

153 36 8 years ago

nl• 8 years ago

Probably the failure I consider the biggest is that I spent most of my PhD trying to solve supervised learning for computer vision using unsupervised feature learning methods, and was caught totally off guard when Alex, Ilya, and Geoff won the ImageNet contest with purely supervised methods. I think that in general wasting time writing papers that turn out to be dead ends is the main way that I fail in my own eyes.

Wow, what an amazing story.

And worth pointing out that Goodfellows work on unsupervised feature learning is clearly seen in his invention of GANs.

Edit, and:

I think it’s hard to extract value from negative results in machine learning because it can be so hard to tell what caused the negative result. A negative result might point to something very fundamental wrong with an idea, but it might also just be the result of a very small software bug, the wrong idea of the hyperparameter values to try out, too small of a model, etc.

So, so true. I read comments on HN on how important negative results are in ML and I look at all the things I’ve tried and someone else made work, and all I see is reporting negative results just discourages ppl from trying things.

This is a great interview.

akhilcacharya3• 8 years ago

Good interview. It really surprised me to see a Stanford undergrad fail at anything to be honest, much less admission to a similarly ranked school for a specific PhD concentration they are known for.

I guess in this circumstance it actually was their loss.

ubernostrum2• 8 years ago

It really surprised me to see a Stanford undergrad fail at anything to be honest

Why? The impression I've gotten from a lot of time in industry is that the average Stanford student/graduate is, well, average.

akhilcacharya3• 8 years ago

Because Stanford is incredibly selective, and is one of the 4 premier CS institutions in the world?

I dunno, as someone that went to a public school with a 50% accept rate it seems difficult to imagine “academic superhumans” struggle with anything, ever.

samfisher83• 8 years ago

If you are Indian you need a 1500+ + 4.0 + good extra curriculars and there is a very good chance you are not getting in. It might similar for other folks, but I don't know. So if the people getting into Stanford are average not sure who you are comparing them to.

laurieg2• 8 years ago

I did my undergrad at Cambridge University and I was surprised at how normal people were. Of course, there were a couple of utter geniuses, but they were the exception.

I think an undergrad friend of mine put it best: "Cambridge doesn't take the top 1%, it takes a certain type of person from the top 10%"

tonyedgecombe2• 8 years ago

When my son interviewed there one of the tutors told me he thought they should abandon the interviewing and select at random from a pool of the top 10%.

icc97• 8 years ago

How many of your friends had anything less than 3 A's (A* since 2010) at A-level?

There's 424,000 students got into university [0] and Cambridge enrols 3,480 out of 17,000 applicants [1]. So they don't take even 1% and only the top 4% of students even bother applying.

Edit: I suspect that 3,480 includes overseas students which makes the the percentage of UK enrolments is even less.

Further [1]:

In 2006, 5,228 students who were rejected went on to get 3 A levels or more at grade A

Perhaps being surrounded by the top 1% makes you feel very normal/average, but there's certainly nothing normal about the Cambridge applicants and your figure of 10% definitely isn't correct.

I got accepted at Imperial with predictions of ABC (+ no extra curricular) to do Maths and Computer Science there's no way Oxford or Cambridge would have looked at me without 4 A's + a significant amount of extra curricular activities.

[0] https://www.theguardian.com/education/live/2016/aug/18/a-lev...

[1] https://en.wikipedia.org/wiki/University_of_Cambridge#Admiss...

microtonal2• 8 years ago

That was a nice read, thanks!

In general, I don't know how I feel about these articles, they seem to be trending among my Twitter friends. I know that these interviews are meant as an encouragement. To show that even the great people of our field had to struggle, so we should not give up when we are still struggling.

However, these stories can also be understood negatively - if you didn't make it, you probably did not work or try hard enough. What makes this effect stronger is the selection bias - we do not see articles of the other 99% that work their tails of and do not become big in their fields. With a society that focuses on being the best, 'making it', etc. it leaves a lot of people feel inadequate, incompetent, or even depressed, even when they are big net contributors to society.

E.g. think of the 50-year old professor a moderately successful academic career. He/she is educating hundreds of students and having a big positive impact on their lives. But they are seen by others and themselves as failed academics who were not good enough to make it.

(I don't know what the solution is.)

AndrewKemendo2• 8 years ago

Well there isn't a solution because the metrics for "failure" and "success" are fuzzy. In my mind, someone is a "success" if they accomplish the challenge of being the person they see themselves as/want to be. It's way harder than it sounds.

So while there are certainly some people who, as Pickard said "made no mistakes but still lost," there are others who didn't put in sufficient effort and failed as a result.

I'd imagine that there are more in the latter camp than in the former. In your example, the professor is only a failure if their goal was something other than moderately successful academic.

tonyedgecombe2• 8 years ago

I suspect the whole goal thing is a problem, setting some specific outcome you expect to reach when you have no idea what obstacles you might face on the way is setting yourself up for failure.

RobertRoberts• 8 years ago

(I don't know what the solution is.)

Is it possible that stories like this are supposed to affect everyone differently?

-#1 Take it as encouragement => succeed, and be "happier", and share the success.

-#2 Take it as encouragement => Fail (and possibly move on to something more appropriate for you, and you are all the wiser for it)

-#3 Take it negatively => Quit at something and move on to something else. (see #2)

-#4 Take it negatively => Succeed anyways despite the negativity (see #1)

Maybe I missed an idea here, but how could it hurt anyone to share this information?

-#1 Take it as encouragement => But fail excessively the rest of your life?

-#2 Take it negatively => Give up and never try again?

It seems the positive outcomes outweigh the negative. And the negative outcomes don't seem reasonable to blame on an article.

yial• 8 years ago

I feel the story of his professor who responded “why do you have an ‘a’ in my class?” Is so incredibly relatable. It seems that sometimes; it’s far to easy to do in some “luck”.

degrews• 8 years ago

I'm getting a 404 Not Found. Anyone else?

brainfog• 8 years ago

From the top level site:

This account has been suspended. Either the domain has been overused, or the reseller ran out of resources.

faded242• 8 years ago

... That's how he fails.

WordSkill• 8 years ago

It has been down for a couple of hours.

I mentioned that in a comment here and, for reasons I don't understand, was immediately downvoted. I deleted the comment in case there is some sort of HN 404 taboo I'm not aware of.

montyf• 8 years ago

I deleted the comment in case there is some sort of HN 404 taboo I'm not aware of.

Never do this. If someone doesn't like your comment, then you are just letting them win by removing it. That's exactly what they want: for the comment not to exist. Don't give them the satisfaction; people who engage in censorship practices don't deserve it.

dragonwriter• 8 years ago

I mentioned that in a comment here and, for reasons I don't understand, was immediately downvoted. I deleted the comment in case there is some sort of HN 404 taboo I'm not aware of.

It's good to reflect on whether a comment that you posted that is downvoted really is problematic in some way, and if you see a real problem with that reflection to delete or edit it, but you shouldn't delete a comment when you don't understand a downvote because, frankly, lots of good posts get stray early downvotes, and that usually gets corrected over time.

veronikach• 8 years ago

I'm the owner of the website (just made an account but @consp can confirm). Basically my webhosting provider didn't like all the traffic despite me having bought extra bandwidth. I'm hoping to get it fixed tomorrow

In the meanwhile here is the post: (link: https://web.archive.org/web/20180505190115/http://www.veroni...) web.archive.org/web/2018050519…

ericliuche• 8 years ago

Getting a 404, the top level site has the error

"This account has been suspended. Either the domain has been overused, or the reseller ran out of resources."

Here's an archive link

https://web.archive.org/web/20180505190115/http://www.veroni...

consp• 8 years ago

The 404 is caused by the provider expecting less get requests than GB's bought. The author (sitting next to me) actually bought more traffic but the company decided to just stop after a number of GET requests.

quasimodem• 8 years ago

Oh I thought the 404 was the joke.

wyclif• 8 years ago

The author clearly needs a new provider.

ivan_ah• 8 years ago

The Deep Learning Book by Goodfellow et al. is amazing. I found the writing to be very down-to-earth and it explains concepts without too much jargon. http://www.deeplearningbook.org/

colmvp• 8 years ago

I can only assume his answer to question 11 is in response to the very public discourse between him and Jürgen Schmidhuber.

amai• 8 years ago

agentofoblivion• 8 years ago

I think the most likely cause of his success is not related to his talent, but his focus. When I was choosing a college, I wasn’t thinking about advisors. It certainly never occurred to me to find the best guys in the field and try to get in under them. It just sounds too far fetched to be a real possibility. But it is a real possibility, of course, and the people that identify that and go for it have a remarkable outcome of accumulating advantage. It’s like compound interest. Go to the best school you can. Get under a top advisor in grad school. Publish in top journals. Work for major companies. At this point you’re surrounded by the best and have become the best. You have to know what you want early, and then optimize your ass off. I was late to the party, but the last couple years have been huge successes for me relative to previous parts of my life, and that started with saying to myself “start by applying to the very best and working down from there.”

How I Fail – Ian Goodfellow

Related Stories