Nate Silver to would-be data scientists: Don’t sweat the degree, do the work

http://gigaom.com/2013/09/30/nate-silver-to-would-be-data-scientists-dont-sweat-the-degree-just-do-the-work/

by cdl • 13 years ago

39 36 13 years ago

gigaom.com

jacques_chester2• 13 years ago

Original source: http://blogs.hbr.org/2013/09/nate-silver-on-finding-a-mentor...

cruise022• 13 years ago

Is it just me, or is the gigaom article basically blogspam?

jacques_chester2• 13 years ago

Saying that something is blogspam can be surprisingly controversial hereabouts. I find it safer to passive aggressively link to the original source.

clubhi• 13 years ago

This is the same with just about everything. When I started learning Clojure I forced myself to stop reading.

carlob2• 13 years ago

There is something that somewhat irks me about the whole domain of data science. I still can't quite put my finger on it, but I think it's a mixture of lack of rigor, handwaving arguments.

Maybe if it were called data journalism, or data storytelling rather than science I would somewhat lower my expectations and enjoy it more fully, but in my experience doing science and doing data science are not really close.

jonahx• 13 years ago

I think you could make the same argument about computer science, especially relating it to the actual practice of professional programming, which resembles pop culture more than science.

And while both fields have mathematical underpinnings, the foundations of data science -- probability and statistics -- have deeper roots and a longer history.

I guess I'm saying you shouldn't impugn the field itself because of its popular practice.

carlob2• 13 years ago

I don't think what most programmers do is computer science, more like software engineering.

ig1• 13 years ago

Data science is about building testable models, which fundamentally is what science is about.

What probably makes you uncomfortable is that Data Science (unlike most scientific fields) doesn't have a requirement for the models to make intuitive sense to humans.

pvnick• 13 years ago

I agree whole heartedly. When I first started playing around with data, despite my lite statistics background, I had no idea what I was doing and no amount of education would have prepared me for noisey, real life data sets. I had to become proficient with SQL and a certain type of problem solving and hacker mentality before I was able to do anything useful.

What's interesting now though is that since returning to college and taking more advanced courses in probability and statistics, I now have a "practical" background that allow me to see real life use cases for almost everything I'm learning (oh wonderful Bayes theorem!) that is a huge advantage.

kartikkumar7• 13 years ago

I find the don't-go-to-school-just-do-it mentality that pervades tech blogs, article, discussions a bit short-sighted. I fully appreciate the importance of getting "hands-on experience" in addition to book learning, but what I don't get is the black-and-white perspective that formal education gives you NO EXPERIENCE WHATSOEVER.

As a PhD candidate, my perspective is quite definitely skewed, but my experience in school so far has not given me the impression that all I've been doing is book learning with no transferrable, real-world skills to work with data. I've had to work with plenty of data sets during my Masters and PhD research. I firmly believe that the experience I've gained from applying what I've learned in the classroom and from surveying existing literature on data mining and statistical methods stands me in good stead to tackle "real data" (whatever that means).

Maybe I've got the wrong end of the stick, but I think that something more than the "odd stats class" can be of value to a budding data scientist ...

ThomPete5• 13 years ago

Don't you find it odd that education start with the abstract to have you end up in reality.

Data science is not about well structured data, it's not about learning what others do. I

It's about finding meaningful information through experimentation with data and for that all you really need is a lot of experimentation, a creative mindset and the ability to extrapolate meaning and correlation out of seemingly uncorrelated data.

First experiment, learn to play around, explore, then apply theory.

kartikkumar7• 13 years ago

Actually, I don't think the process of going from abstract to reality is that odd. The reason I say that is because I think understanding the abstract prepares you to be flexible. In terms of data science, the way I see this is that by experimenting without having a grounding in the algorithms, technologies etc. means that you can only really ever get good at the specific implementations you're working with. The abstract understanding is precisely what allows you to be versatile in adopting new technologies and seeing the implementation for what it is: one concrete version of the underlying abstract ideas.

There's nothing that says that formal education only prepares you to work with well-structured data. In fact, I think that in grad school you are challenged to work at the cutting edge almost all the time. Granted, a lot of classroom examples at the undergrad level are perhaps trivial, but my experience is that grad school doesn't by definition only serve to prep you with pointless examples.

It's about finding meaningful information through experimentation with data and for that all you really need is a lot of experimentation, a creative mindset and the ability to extrapolate meaning and correlation out of seemingly uncorrelated data.

I totally agree, and I believe that school is not at odds with this. I'm working in the fields of planetary science and space engineering at the moment, and I am constantly stimulated to experiment with data, be creative and understand correlations.

Maybe for some reason my experience is just not a good 3-sigma example ...

mattlutze2• 13 years ago

(I'll admit to skimming up front)

I didn't get the impression that he was suggesting people interested in Data Science should forego theory and lecture altogether.

His argument for building applied skills first, was that it would motivate academic pursuit later (as well as being immediately applicable). If you're already reading for your Ph.D. we might assume you're not at a loss for motivation :-)

(okay now I went back and reread)

I also get the impression he's talking to an application-focused audience. 16 year olds looking at colleges, or 20 year olds considering a focus for a masters, are probably not the target audience of the Harvard Business Review.

kartikkumar7• 13 years ago

Yea, I appreciate the fact that he isn't suggesting that people abandon school althogether. I think what irked me is his suggestion that school is maybe useful for the "odd stats class", suggesting to me that he doesn't believe that there is value beyond that.

Perhaps my reading of the blog post isn't entirely correct. I agree that he's not making any statement directly to dissuade people from going to school. I just think that it's important to note that my experience of being in school is that it has given me freedom to be creative, experiment, play around with new technologies, and ultimately build applied skills. Perhaps I'm in the minority that feels that way ...

EDIT: After reading the full blog post, I can see that his view is more nuanced than comes across on GigaOm.

hooande• 13 years ago

It's worth pointing out that Nate Silver is a rare individual. He's been wildly successful with data science despite not having a deep academic background, which is not the common case. His story is a useful data point though.

Do you need multiple degrees to master data science? Hell no. The practice of statistical inference requires only a few key skills. The important part is understanding how to think in terms of probability and algorithms, which doesn't require a rigorous understanding of proofs and formulae. An advanced academic degree is useful but not required. The world is looking for people who can consistently get the right answer, regardless of their background.

The other side to this is that a degree will be required to work for wall st, google, harvard, etc. Degrees still matter and social signalling and so on and so forth. We all can't be as well known and popular as Nate Silver and at some point institutions need a way of establishing that a particular candidate is truly qualified. So if you're not a messianic ml rock star, sweating the degree might be a good idea.

The beauty of a purely empirical craft is that results are the only arbiter of success. Anyone can get the right answer. It might be awhile before society catches up to and utilizes this powerful idea, but the hiring competitions on sites like kaggle are a great start [1]. Until then the best case is being the next Nate Silver.

[1] http://www.kaggle.com/c/facebook-recruiting-iii-keyword-extr...

mathattack• 13 years ago

Like anything, real world experience complements academic background, and over time the credential means less.

But as a thought experiment... All things equal, who wouldn't hire someone with 2 years experience + a masters, over someone with just 2 years experience, or just the masters.

I like to see something that signifies intelligence and curiosity, and then something that shows someone has done something with it. Usually this is a combination of academics and work. It can be shown just with work. It's hard to show it just with academics.

wikiburner• 13 years ago

The practice of statistical inference requires only a few key skills.

Just curious, what do you think they are?

mattlutze2• 13 years ago

“… Getting your hands dirty with the data set is, I think, far and away better than spending too much time doing reading and so forth,” Silver said in a Q&A with HBR’s Walter Frick.

I'm in a data science class (hi carlob) right now. This is exactly how we've approached the subject. There were a few readings the first couple weeks to get people up to speed on Python (or scare them away if they didn't have any coding experience), then our homework problem sets have jumped straight into real-world data scraping and modeling.

It's been a fantastic way to start learning the field, because after just 3.5-ish weeks I already feel like I have the tools I need to start exploring and fitting rudimentary models to the information I'm collecting.

It's exciting and enabling, and I think that's definitely a strength of doing while learning.

kyllo2• 13 years ago

I feel like Data Science is not a real academic field, and it's not something you need a college degree in. The programs are so new and rudimentary, the universities don't really even know what to teach the students in an "Intro to Data Science" class. But it's a buzzword in the media, and there's employer demand now, so I think that universities saw the $$$ and just ran with it.

It's really more of a technical skillset and toolset for statisticians and other actual scientists who use statistics to test their hypotheses. Make Intro CS, Relational Algebra and Set Theory, and Computational Statistics electives for Bachelor of Science degrees. This really doesn't need its own degree program, and neither did "business intelligence" back when that was the buzzword for this. Universities aren't for "latest tools and fads" training.

dragonwriter2• 13 years ago

I feel like Data Science is not a real academic field, and it's not something you need a college degree in.

Its an emergent academic field in the way that bioinformatics was a few years ago; "data science" is a name for something that's been taught in less breadth and depth as parts of other academic disciplines (including parts of the "research methods" curriculum in the life and social sciences, parts of statistics, parts of computer science), but certainly its as legitimate an area of focus as the disciplines its that parts of it have been hosted in. There's more than one way to divide up intellectual pursuit, and a number of ways that have cut across the previously predominant divisions have been coming to the fore recently. This is, really, a good thing.

kyllo2• 13 years ago

Yeah, I guess what I'm trying to say is that Data Science is really a "horizontal", not a "vertical" in academia, to borrow business jargon. It's an interdisciplinary research skill set that students and scholars across academic disciplines should be trained in. As such, I'm not convinced it needs its own degree programs, though.

mrcactu5• 13 years ago

most people don't really think logically or quantitatively. whatever math people learned in high school and college was forgotten when trying to make adult decisions. and they didn't even get to the good stuff.

data science is successful now because they are applying quantitative methods to the social sciences -- problems most people think about. the average educated person can understand by using the magic of "machine learning" we can solve problems in politics, health, social issues, etc.

Nate Silver's lack of a math background works to his advantage. he realizes the bar for quantitative understanding in the public sphere is very low. whatever little bit he does, he has to explain to a room full of drunk people in 30 seconds or less. he is very good at this.

neither employers nor the public want to admit that at the end of the day SOMEONE has to be responsible for these calculations. i have a master's degree in math -- which nobody gave a hoot about (see item 5 in this article http://gigaom.com/2013/04/16/how-to-hire-data-scientists-and... ) we weren't taught how to communicate our ideas and results to the public -- in fact we were discouraged from it. that is difficult to unlearn.

i've heard of attempts to "democratize" machine learning. http://blog.bigml.com/2013/03/06/democratizing-machine-learn... articles like these are usually written by companies with a product or book to sell or distribute. what needs to be democratized is engineering based problem solving - using the powerful computers we hold in our pockets, our phones.

slurry• 13 years ago

Nate Silver's lack of a math background? Why does this keep getting repeated. He has a BA in economics from one of the most quantitative and rigorous programs in the country, plus some work at LSE. I'm sure he has more formal math and statistics education than most self-labeled "engineers" on HN.

geebee• 13 years ago

Here's a link to the university of chicago's econ major requirements.

http://collegecatalog.uchicago.edu/thecollege/economics/#pro...

Looks fairly rigorous. Can't say for sure, but it does appear that while you are encouraged to take the most rigorous calc sequence (typically required of math, hard science, engineering majors), econ majors are allowed to take an easier calculus track. That's pretty typical of econ undergraduate degrees - many universities do provide a slower, easier calculus track for certain majors (typically life science majors or social science majors).

I have no idea what track Nate Silver took, and of course I can't speak to the "self-labeled" engineer. I'd agree that an econ from a reputable school has some mathematical rigor, but not quite at the level required of a typical math, hard science, or engineering major.

Either way, I would agree with you that it is silly to say Nate Silver doesn't have a math background - clearly he does.

abbazabba• 13 years ago

Is there anyone around that "does data science" for a living that can talk about what it is they do exactly?

I'm under the impression that "data scientist" is as ambiguous a term as "designer". What's the minimum amount of math or statistics required to be labeled a data scientist? And what's the minimum amount of programming too?

Nate Silver to would-be data scientists: Don’t sweat the degree, do the work

Related Stories