mistermcgruff

Churchill also took a 2 hour nap daily. The man would change into his damn pajamas to do it. Night time sleep is not total sleep. Let's stop glorifying folks who supposedly need so little sleep...they may be getting it other ways.

I wrote a technical book for Wiley. 12.5% royalties for paper sales. 25% for book. Negotiate the ebook rate.

The main thing that surprised me (which shouldn't have) was the complete lack of marketing Wiley did. You write it, you find the tech editors, you edit it. They assign someone to you who basically bugs you to turn in chapters. I had the cover designed myself so it wouldn't suck. Then it comes out and you are the one who has to market it. but for me it was fun just to do it.

Can it play "drugwars"? https://en.wikipedia.org/wiki/Drugwars

All I know is that I spent two years deep in abstract algebra and other proof-heavy classes, and it was some of the best training in rigor and thoughtfulness I've ever had. Wouldn't trade it for the world. But, then again, most recent grads weren't math majors. I think my university of 30k graduated about 15 math majors a year.

I used to have a breadfruit tree in my yard in Tennessee. We would take the fruit as it fell and line it up in the street and watch cars run over them. Seeing them crushed was pretty satisfying (I was 9 years old at the time).

Then one day a car lost traction on our breadfruit gauntlet and the driver got terribly pissed. Haven't played with the fruit since.

This has not improved. However, instead of using screen in a terminal to do a long create table query, I'll now just kick it off in pgAdmin and yank my ethernet. This causes the query to just hang out there indefinitely until it's done. Stupid, yes. But it's kinda fun.

Yeah. Autism is diagnosed by symptoms and the symptoms have changed over time. It used to be that an individual had to actively avoid social contact. Now they just need to be socially impaired even if they seek contact. As the symptoms have changed and as awareness has increased in the medical community, so have diagnoses increased.

A semester of linear algebra (or just a willingness to Wikipedia a few things) plus Excel experience is all you need.

That said, the book does require a lot of effort, because the techniques are worked through step by step.

But once you learn all the guts of the algorithms, you never have to implement them again! The last chapter moves the reader into R package land with the confidence that you now know what those packages are basically doing and what to watch out for.

If you've got a college semester of linear algebra under your belt (or equivalent) and are pretty good with Excel, then the book is a good fit. Even the algebra can be optional if you're willing to use wikipedia liberally. I don't take for granted that the reader has a lot of background.

That said, there are parts in the book that are really quite hard. Hard in that they just take time to work through. Because the book is about learning all the steps that go into training models and doing analyses from scratch. But once you do it all from scratch once, you don't necessarily have to ever ever do it again.

It's taught in Excel for learning purposes, and then the last chapter moves you into R. Literally, the Holt Winters forecasting chapter of the book is 50 pages while in R it's the forecast package plus 3 lines of code.

Hey, author here. It does sound kinda bigoted, doesn't it? Lemme clarify. I would really love for more people to learn this stuff. Truly. That's why I wrote a book to teach it to folks in Excel. What freaks me out a bit is that in order to enlarge the ML pie, a lot of vendors are trying to democratize machine learning not by teaching it to more people but by putting the gun in hands of folks who don't actually understand how supervised machine learning works and where it can go wrong.

I don't think people need PhDs to learn or do this stuff. But I think they need more than a "machine learning made easy!" app and a gung-ho attitude.

I feel that I'm halfway between main street and an ivory tower...not sure where that puts me. The upstairs bar at a pizza parlor?

I make Excel choke regularly. For example, copy-pasting a complicated vlookup down 1 million rows takes forever.

Sounds more like a classic optimization model (operations research) than like an AI model. Great case study though. I wonder why they used a GA and custom stuff instead of off the shelf software from gurobi or IBM OPL etc.

Another way to get free internet in hotels is to tell them it was shitty internet when you check out. It always is shitty (just try watching a movie on it!). They always credit it back to me.

Tufte is only one perspective. And he's irritatingly preachy at times. Hell, this is his favorite website: http://forecast.weather.gov/MapClick.php?lat=33.769373434837...

Are the folks at NOAA non-poseurs? Is the height of design just to vomit all the information on a single canvas, because Tufte thinks "our eyes can handle it?"

I entered college exempted from one year of classes. Rather than graduate in 3, I used those credits to graduate leisurely in 4 with a pure math degree. Math majors had the lowest gpas on campus and by far the hardest workload. Abstract algebra is tough! I don't regret my pace... It allowed me to play bar trivia every once in a while.

Not seeing HBO Go. That means no GoT on demand :-(

Was wondering the same thing about the Aphex Twin. Is that legal?

I like to think about whether a problem is inherently antagonistic when deciding whether data/models/etc. can automate it well ("eat it" in the words of the article).

Not antagonistic -- predicting whether someone will get lung cancer...they're probably not going to falsify the data from their physical, etc.

Antagonistic -- detecting network intrusions or predicting phishing attacks. These folks fight back, i.e. disguise, adapt, hide

In the latter case, a human component to a decision task (detection, labeling, etc.) will always remain.

So then when it comes to investment decisions, let's say data plus supervised ML becomes the name of the game. Start-ups will learn the model features of a business that gets investment dollars. They'll design themselves specifically to those features and in a sense dupe models for a while until the models are adapted. In this way, the human component can't be left out.

Of course, it doesn't take a model to get duped. A lot of companies might hire a bunch of phds and data scientists right before being acquired, because each hire bumps up the value of the company, etc. These moves are taken to increase a valuation rather than actual value and that's antagonistic in much the same way hiding spam is.

That really stinks. I can't imagine how annoying that must get. I work at an email service provider, and I deal with about two billion email addresses from all around the world. Apostrophes are certainly a consideration when working with emails, loading them to and from databases. But the spec for what's a valid email is very very wide. Lots of special characters including apostrophes are technically valid, so if I'm doing an analysis that requires email as a unique key, but I don't actually need to know the addresses, I'll just hash each address. It's lazy, but man does it solve a lot of special cases.

Here's a reponse to Tufte in defense of PowerPoint, which was on HN a few weeks ago: http://www.john-foreman.com/1/post/2014/01/defending-powerpo...

The only problem is when you send ships to mine other planets, there may be eggs with face-huggers in them on those other planets.

I'm an R person, but I used SAS in a past life, and I have to say, this is very wrong. SAS is nothing at all like an improved Excel in look, operation, or user-base. It can do much of what R can do (build forecast models, ML models, even OR models), but it just looks very different.

And the licensing model will make you pull your hair out. I ran into issues both with geography (not being allowed to use my license on a project in another country) and functionality (hey, that's a cool PROC...wait, I don't have the license to call it)

I think R and python will win the day, but it's not because SAS is anything like Excel. And there are a shit ton of "serious" data analysis people using SAS. They're just all in the enterprise. Every Fortune 500 company I've worked with used SAS except for one (who used R).

I get it weekly. It's up to date. Very handy too.

This had me crying I was laughing so hard. In an airport bar. People were staring.

It may be obvious, but throughout Tufte's work he ignores that distinction and conflates the two. So it must not be obvious to Tufte.

I just published a data science book: http://amzn.com/111866146X

I went through Wiley, which is a very traditional, large publisher (my editor is the same as Mitnick's and Schneier's). You seriously limit the royalty rate you're going to get this way, but you're going to extend your reach and marketing ability. So it's a balance, and you need to figure out whether reach or per-book royalties are more important. If you have a small audience and you know you can reach each and every reader yourself (maybe on Twitter or HN or Reddit), then self-publishing may be the way to go.

I chose Wiley, because the audience for my book is broader than most data science books (since it's spreadsheet based), and I wanted to get it in front of your average BI analyst perusing at Barnes and Noble.

The big publisher also did a great job with layout, the book feels great with its matte finish, I had two paid tech editors, a regular editor, a project editor, and a bevy of proofreaders. Big help, that.

But don't be mistaken...the onus is still on you to market the book as well. And it's going to take a lot of marketing to make this a viable career (or even a viable supplement to your income). It's an incredibly taxing way to make a living I'd imagine, and I'm super happy to have a great day job.

Rather, my reason for publishing was that I saw a need for the book, and I knew I could write it. And it took a full year from starting to get it out. So it took a year of hard work, and it'd be a miracle if I cleared twenty grand on it. Not the most lucrative endeavor, but extremely personally satisfying.

To answer some of your other questions, the publisher released in both paper and ebook. Pictures in the book were a real pain in the ass. I spent forever screencapping and re-screencapping spreadsheets. Ugh...

This is amazing.

I agree that the notation is tough. Especially for folks who need to learn to do this stuff as a one-off and not as a career. That's why I wrote a book that teaches the algorithms with nearly no math notation while trying to be very rigorous: http://www.amazon.com/Data-Smart-Science-Transform-Informati...

Very cool. Naturally, scraping out of HN is gonna provide a few wonky teasers, but a great idea nonetheless. Just submitted my book per the site's instructions for kicks http://www.amazon.com/dp/111866146X

Yeah, google drive spreadsheets are only good for sharing fairly static small sheets. I just wrote an intro data science book that uses spreadsheets to demonstrate certain algorithms (simplex, kmeans, modularity, LOF, etc.) And in the book I warn the reader to not use Drive. Slow, clunky, and the Solver implementation is garbage. Plug:

http://www.amazon.com/Data-Smart-Science-Transform-Informati...