Helping my students overcome command-line bullshittery

http://pgbovine.net/command-line-bullshittery.htm

by luu • 12 years ago

116 215 12 years ago

akumpf2• 12 years ago

This is like an alternative form of "Yak Shaving." :)

Instead of going super far down a path, following each step along the way, and somehow ending up with a yak in your hands, you're immediately asked to find a yak and you rightfully ask, "what does this have to do with yaks?"

Yes, the command-line is powerful, but it is definitely a huge hurdle for most as they're really just trying to do something else. Well stated PG.

zachh• 12 years ago

This is exactly the problem we're trying to solve with http://bowery.io. There's so much amazing OSS that's hard to set up, making it hard to focus on the actual development/coding/research you want to do.

dasboth2• 12 years ago

At first glance this looks amazing! I'll definitely have a play with this, thanks for sharing.

williamstein• 12 years ago

Similar with https://cloud.sagemath.com, except in the domain of computational mathematics. People like you and I deal with a lot of the BS, so it's already setup for other people.

buzzkillr2• 12 years ago

Using sage math now in a class and it's tops. Thank you for your work

jaderobbins12• 12 years ago

I feel like any field of research has a base set of knowledge and skill sets required to do high level research. One could say that biology research is filled with "Microscope Bullshittery" or Paleontology is filled with "Fossil Digging Bullshittery".

A base skill of doing computer science research is programming.

Can you program without computer science? Absolutely. Can you Computer Science without programming? I would say no. Being able to look at and understand how to use a language or library is just something required to get to the last tier of computers science knowledge. I think every researcher would love to get rid of their bullshittery, and often they have lab technicians or interns do it for them but in the end they all had to pay their dues and have to know it in order to mentor and help those below them.

saidajigumi2• 12 years ago

The author specifically calls out that he's not talking about programming, per se. He's talking about the skill set of wrestling useful free software packages to one's own aims:

So perhaps what is more important to a researcher than programming ability is adeptness at dealing with command-line bullshittery, since that enables one to become 10x or even 100x more productive than peers by finding, installing, configuring, customizing, and remixing the appropriate pieces of free software.

I'm torn about this article. Clearly this researcher, in his role as mentor, has identified a skill gap that's hindering his students. And it's perhaps even a problem that the software community can ease the pain of. But many of the things he lists in passing get down to fundamental tools of software work: version control, package management, data manipulation, etc. Yes, the usage of these things on the command line tends to be "arcane", but that's because each is encoding its own problem domain. And if you're going to be working in software in any non-ivory-tower capacity, you'd better know this stuff.

I've dealt with this kind of problem numerous times before in various contexts with workflow tooling. I.e. a single (usually) command-line tool that neatly encapsulates the most common development use cases to reduce learning curves, cycle time, and errors. These can be phenomenally successful if done well, but if the context doesn't define a workflow (e.g. student A vs. student B's research ideas) then there's no easy way to encapsulate the user's problems.

diminish3• 12 years ago

Clearly this researcher, in his role as mentor, has identified a skill gap that's hindering his students.

This is a well known pre-course prep step, part of a bigger to do list for all teachers to make sure the tools you suggest to the students are bundled to be setup in an easy way for your target audience.

DontGiveTwoFlux• 12 years ago

I think the biggest issue is that beyond the setup phase of development, these tools don't get used by the new researchers the author works with. If I'm developing a new program, having to run ten programs I've never seen before just to get started can be frustrating if I won't be actively using as them I work.

Programmers should learn the tools to stay efficient. Version Control, build tools, etc. are priceless. But if you force feed too much at once, nothing will stick. Couple that with what for some may be their first time on the command line, and you have a recipe for bullshittery.

I'm having a hard time understanding other commenter's grief that explaining git to someone who's never typed "ls" before is anything less than bullshit to slog through. These things are best learned one or two at a time.

mcguire10• 12 years ago

He also goes on to say,

"Throughout this entire ordeal where I'm uttering ridiculous epithets like 'git pipe fork pipe stdout pipe stderr apt-get revert rollback pipe pipe grep pipe to less make install redirect rm rm ls ls -l ls tar -zxvf rm rm rm ssh mv ssh curl wget pip,'"

In other words, "ridiculous epithets" seems to be equivalent to telling the machine to do something. Have you got a way to get git to control your source without actually invoking git?

Workflow tooling can indeed be incredibly useful, but the context isn't the only requirement for success. If something underpinning that tooling changes or breaks, someone is going to have to understand what happened.

The people who regard that understanding as "ridiculous" are the worst people to work with and to my mind are the primary reasons that this "profession" gets little respect.

loup-vaillant8• 12 years ago

Yes, the usage of these things on the command line tends to be "arcane", but that's because each is encoding its own problem domain.

Not necessarily. I came to believe lately that Git for instance, has beautiful, simple, and powerful core principles… and an unacceptably crappy user interface. A hashed DAG of commits with a few pointers to navigate it, that's great. But the magic incantation that you're required to type on the command line are too complex, unintuitive, inconsistent… and intellectually utterly uninteresting.

Git's core model is the interesting part, the one that will make you a better programmer, or computer user, or whatever it is you want to do that involves version control. But the specifics of the command line interface? That's neither interesting nor a portable skill. "Command line bullshitery" is a perfect term to describe it.

Why I believe that has been said better than I can ever do here: http://tonsky.me/blog/reinventing-git-interface/

Seriously, even "end losers" could use this. I also believe this can be generalised: some software just isn't usable through the command line. For day to day interactive use, it needs a neat, special purpose graphical user interface —Bret Victor has taught us how powerful they can be.

The command line is still invaluable when interacting with other software, or for automation. Then it should be designed for those, not for interactive use. Simply put, it should be an API —which you could use to build your wonderful GUI on top of.

danielweber4• 12 years ago

Paul Graham has talked about the pain of installing software, and every time I have to do it, I always have trepidation. "Is this going to be the time apt-get barfs at me?"

Software installation is still a big pile of bullshit. For the people who spend their time deep inside one ecosystem, it can be okay, but most people have something to do besides live deep inside one ecosystem.

A few weeks ago I was just trying to find a JavaScript minifier on a Linux VM. So I googled and spent an hour digging through various pieces of crap, incompatible versions of libraries, asinine "gem install" error messages, and fun reading Stack Overflow answers saying things like "why didn't you have lib-foo-fuck-you installed already?"

And none of this is valuable for me to learn because in five years all the current package maintenance stuff is going to be thrown out and replaced. Not necessarily by something better (although that's likely the hope, leading to http://xkcd.com/927/ ).

yodsanklai• 12 years ago

Can you Computer Science without programming?

The bullshitery he's talking about isn't programming, but compiling, installing and setting up various software that are still in an early stage of development and aren't user friendly yet. Just something you have to do but isn't related to programming or research.

Somehow, I agree that an all-around computer scientist has to know how to deal with this, but it's probably not the best way they can spend their time.

mcguire10• 12 years ago

One of the neat things about computer science research is that researcher B's work is frequently based on researcher A's software and A's software is "academic", which means that it actually works on exactly the three examples in A's dissertation.

Curmudgel2• 12 years ago

Well, yes and no. If you want to look at something very small, you have to learn how to use a microscope. It's not that hard to learn how to use a simple microscope, and you can make adjustments by thinking about the physical principles, which are universal. And if a certain type of microscope isn't suitable, you can always change or someone can design a better one.

The *nix command line is not based on physical law. There is no mathematical or physical reason that human-computer action has to be through an underpowered, user hostile design that requires lots of unnecessary memorization. People use it because everyone does. You can come up with a better design, but you'll still have to use the old design if you want to interact with anyone else.

Edit: By "underpowered", I mean that passing byte streams and parsing them is less powerful than passing objects. And this is made worse by the fact that different commands have different output syntaxes.

sergiosgc• 12 years ago

(...) underpowered, user hostile design that requires lots of unnecessary memorization.

User hostile I can understand. Underpowered is just plain false. Parsing log files, relating data between them and aggregating results is a common task for any decent sysadmin. I know no interface as powerful (measured as the ratio of information quality over time spent) as the Unix command line.

Many powerful interfaces have ultra steep learning curves. The Unix shell is one of them. Steep learning curves are a flaw, of course, but they do not invalidate the other qualities of the interface.

jnbiche• 12 years ago

an underpowered, user hostile design

In your opinion, of course. Many of us find the * nix command line not only elegant and highly-productive, but actually enjoyable.

In fact, the only Linux command line tools that I did not immediately start using in a highly productive manner are `tar` and `find`, arguably two of tools that least abide by the ideals of * nix command line tools (I've since gotten used to `find`, but `tar` may always send me searching for my personal wiki's corresponding entry[1]).

If you want to argue that there are better ways to program on the console, I grant that it's possible -- although as mentioned, I find the composability of * nix tools to be an almost magically productive approach.

But to condemn anyone to a life of GUI tools is not only going to drastically increase their chance of developing carpal tunnel syndrome, but also inevitably will slow down their work flow -- often drastically. It's simply not possible for even an experienced user to point and click with a mouse as fast as an fluent typist can issue commands on the terminal.

1. http://xkcd.com/1168/

jerf2• 12 years ago

If the command line were underpowered, it would have been abandoned decades ago. It's still around precisely because it's incredibly powerful, and nobody has come up with a suitable replacement that doesn't involve sacrificing a lot of power.

Further, all of the "No, obviously this would work much better!"s have been tried already. Dozens of times, if not hundreds. Some of them you can even download right now. Nevertheless, the command line persists. It may not be perfect in every detail, but it's far harder to exceed than it seems at an initial, frustrated glance. If you hope to someday so replace it, it would be wise to understand exactly why that is, lest your effort completely fail in exactly the same way as so many prior attempts.

ska2• 12 years ago

Part of the reason that people still use the *nix style interface is that the large number of people who thought they'd come up with a better design were actually wrong about that. It's harder than it looks.

dllthomas10• 12 years ago

You can totally "Computer Science" without programming. It's a big field, and a big portion of it is pretty much math. You probably cannot "Applied Computer Science" without programming.

Dewie2• 12 years ago

I initially thought the same thing. But then I considered that many of the theoreticians probably write their papers in LaTeX.

jaderobbins12• 12 years ago

I just replied in the context of the article. I'm assuming if you are doing math proofs all day you probably aren't dealing with "command line bullshittery".

tw04• 12 years ago

I find it sad, that in an academic setting no less, a PROFESSOR is telling his students it's BULLSHIT to know how the systems that are vital to their work actually function. He sounds like the guy that goes to jiffy lube because he doesn't know what an oil filter is, and thinks that it's "bullshit" that anyone does. I just want to turn the key and go! That's great, but as an academic you should also want to know what the fuck happens when you turn the key. And encouraging your students to devalue knowing how things work under the covers... I think he needs to find a new line of work.

JoeAltmaier7• 12 years ago

Be fair. Every class isn't "How to configure a VM and toolchain" class.

djur• 12 years ago

Shouldn't computer skills courses be prerequisites for programming courses? He's right that setting up the needed software shouldn't be an impediment to completing work in a programming, but that's because the university should be preparing the students with those skills before they get to his course.

If you're going to need to configure a VM and toolchain for many of your subsequent courses, it makes sense to start off with a "configure a VM and toolchain" course.

JoshTriplett• 12 years ago

The very first class in the CS curriculum I took was "Intro to UNIX". Every other class in the curriculum assumed that as background knowledge. It's completely reasonable to expect a configured environment.

akumpf2• 12 years ago

I understand the perspective you're taking, but I politely disagree. I think you're missing the bigger point here.

Let's say someone is interested in baking a cake. There is a LOT you can learn, spanning general baking techniques, chemistry, design, art, tasting, etc. But if your immediate response is "we need flour, so go plant some wheat and wait a few months," they would likely lose interest.

Teaching people to plant and harvest wheat is awesome, but for most people it probably shouldn't be the first thing you're met with when you are trying to learn how bake a cake.

Kalium13• 12 years ago

If you're studying "Baking Science", which covers everything from the beginning to the end, starting with "Let's grind some flour" is a good idea. People studying "Baking Science" need to understand the whole process, rather than believing everything starts and ends with pre-packaged recipes and machines that do everything for them.

That's what's going on here. A "Baking Science" curriculum that didn't impart people with a knowledge of where flour comes from and how it's made would be a joke.

diminish3• 12 years ago

To be fair, unfortunately as of 2014 considering there are 10^2 main disciplines, with 10^4 main research areas, with 10^6 different things to learn and 10^7 researchers building stuff to teach, expecting a GUI button to do those myriad of functions in diverse ways would be naive. Maybe, the primary school education tools are close to what you/he describe in simplicity of use.

matt_kantor2• 12 years ago

The analogue in CS would be to give the students a pre-configured environment if you want to deep dive into a specific topic before teaching how to set things up. If the students are using a known OS, all you need to distribute is a shell script. Otherwise give 'em a Vagrantfile or something.

That being said, you'll want your students to understand their tools sooner rather than later.

fidotron2• 12 years ago

You need to say hello to the canonical example Dijkstra, where arrogance in computer science is measured in nano-Dijkstras, to steal from Alan Kay.

Dijkstra famously observed that computer science is as much the study of computers as astronomy is of telescopes.

pyre4• 12 years ago

That explains why NASA (and other agencies) always seem to have so many issues with unit conversions. It's all just Measurement Bullshitery that they shouldn't have to deal with! /s

dead10ck• 12 years ago

My thoughts exactly.

"But what about the magic of version control, GitHub, pull requests, forking clones, cloning forks, diffing forks, forking diffs, etc., etc., etc.? None of that matters for someone who works with binary file formats (i.e., user culture) rather than plain-text formats (i.e., programmer culture). There was such a disconnect between the two cultures that it was hard for me to come up with responses to these sorts of “why use X rather than Y” questions without sounding either incomprehensible or patronizing."

1. Is he seriously implying that version control is just some arcane thing that Unix programmers do, and that it's bull shit that students have to learn it? 2. If you can't answer students' questions about why these CLI tools are needed, then that does not mean there are no good reasons--it means you don't know, because you don't understand yourself.

"It's comically absurd that the most sophisticated computer programmers nowadays are still using an interface that's remained unchanged since before humans landed on the moon. What the duck?!?"

"Students are starting to grow suspicious: These instructors are supposed to be expert programmers, but their tools look pretty primitive to me."

This is a problem of perception, both on the students and on the professor's part, not a problem of reality. Do you think there's a reason that programmers still use these interfaces decades later? I'll give you a hint: it's not because they don't like change.

recursive3• 12 years ago

1. No, I think he's implying that the user interface his students are required to use is bullshit.

loup-vaillant8• 12 years ago

1. He is implying the students believe it. From their point of view, the CLI does look arcane, primitive, and useless.

2. it's more complicated than that. Yes, if you understand something, you can explain it. No, you can't automatically explain it in 5 minutes. Sometimes, the knowledge gap is so great that you need months to explain it all. In 5 minutes, you can only make claims your students will feel entitled not to believe if they feel like it. If your claims don't match their experience, and you have not demonstrated trustworthiness, they simply won't believe you, then tune out. http://lesswrong.com/lw/kg/expecting_short_inferential_dista...

tptacek2• 12 years ago

Speaking as someone who has changed his own oil more than once: if I needed to know how to do that in order to operate my car, that would indeed be bullshittery. I benefited in no way whatsoever from changing my own oil. It was, as it turned out, simply a very- low- ROI- recreational activity.

The point here isn't that the command line is bullshit. Knowing how an engine lubrication system works isn't bullshit... for a mechanic. But it is a waste of time for a cab driver.

diminish3• 12 years ago

So considering the OP is an "Assistant Professor of Computer Science University of Rochester" please refine your analogy whether his target audience falls closer to become the cab driver or the mechanic?

Edit: quoted below OP's case:

> Write a piece of prototype computer software that demonstrates the feasibility of a novel idea."
> Write a piece of prototype computer software that collects, processes, and analyzes data to produce novel insights about some topic"

calibraxis• 12 years ago

As a professional programmer, I think that prof offers excellent advice. There's a difference between programming-in-the-abstract, and the mis-evolved "ls -l ls tar -zxvf rm rm rm ssh mv". Written by people in real-world dysfunctional settings.

There's alternatives to bash commandlines, to interactively explore/navigate running systems. Take REPLs, SQL prompts, etc.

Profs don't do their students any favors if they let poor design pass without comment. Unix has a history, which you can get if you read early papers from those involved, and users can critique it like they critique whatever website they use nowadays.

pyre4• 12 years ago

Take REPLs, SQL prompts, etc.

That's still "command-line bullshitery" according to him though. Because it's an 'arcane interface' that "hasn't changed since the 60's". When his student wonders why he is using a text-based interface, he doesn't have an answer. Since he doesn't know, then obviously there is no good reason, so It's All Bullshit(tm) is the answer.

mcguire10• 12 years ago

There is indeed a difference between programming-in-the-abstract and Guo's "bullshit"; it is exactly the difference between whiteboard doodles and running code: programming.

jcampbell1• 12 years ago

His point was that it is a bullshit tax that you must pay.

The Jiffy lube analogy is wrong. It is more like buying car insurance, registration, inspection, and filling the car with gas. Those things are "bullshit" that has nothing to do with what I am trying to accomplish with a car, which is going places faster than walking, and carrying more stuff. I must do them, but they have little to do with what I want to accomplish with a car.

rprospero• 12 years ago

While the original article didn't particularly garner my sympathy, I get concerned when people start complaining that others don't know how systems "actually function".

Image that, the next time you sat down to write come code, you're asked for the value of Planck's constant and to solve a couple of Feynmann diagrams for holes in a semi-conductor before you can write a single line. After all, a programmer can't just live with the high level abstractions of machine code and Kirchhoff's rules - she should know how the systems she works on actually function!

Everyone uses some abstraction when they think about writing code. High quality abstractions (e.g. machine code) should be encouraged and things which violate those abstraction (e.g. bombarding memory with radiation) should be avoided. Low quality abstractions (e.g. monads are burritos) should be avoided. Being charitable, the professor seems concerned that poor quality abstraction are dominating and students are being forced to drop to abstraction levels that aren't relevant to their studies.

showerst• 12 years ago

At this point, isn't it more effective to have a virtual machine image with everything set up? Virtualbox runs on tons of platforms, and I imagine you could build a 'CS' box with most of the tools installed and just update it a few times a year.

Granted, that keeps them from learning how to set it up on their own, but his stated goal is to cut out that BS.

I think this would be even more valuable in undergrad, I remember my first few college CS classes spending the whole lab helping people install java on various flavors of windows.

dasboth2• 12 years ago

This was what I was going to say. As an example, a course I did on Coursera recently had an ubuntu VM you could just download and it ran everything that was required for the course. This is probably quite prevalent in CS-related MOOCs and I think in most cases is a great idea.

pmontra• 12 years ago

A course I've taken on Coursera years ago did exactly that. Not that would have been a big problem for me to setup the software we needed (I was on the very same Linux distro anyway) but it saved time to me too and I didn't read all the usual bullshittery (funny word) in the forums.

So Dear Professor, you had to deal with the command line to setup your own machine, do it in a VM and let your students download it.

For extra safety pin the versions of the software your students will be working on (for example Octave) if you're afraid that an automatically updated new version will be incompatible with the assignments and all the other stuff you prepared for your class.

lbearl2• 12 years ago

The bad thing here is that it sounds like these are graduate students, not undergrads. Grad students should understand how to wrangle the command line (if not, how did they get through undergrad?).

jjindev• 12 years ago

I think this is a "stages of moral development(1)" thing. In post-conventional levels of development, all tools would be valued for their strengths. We wouldn't denigrate a tool for belonging to a class of tools.

1 - with apologies to Kohlberg

dmschulman• 12 years ago

"I've spent the past decade mostly leading my own research projects. This meant that I did the majority of the command-line bullshittery and programming to produce the results that led to publications, especially ones where I was the first author. In short, I've gotten very, very, very good at command-line bullshittery."

You mean you learned a new skill and got better at it? Your example suggests that there's only one way to improve your skills when it comes to working with "command-line bullshittery"

lucb1e• 12 years ago

In a linked article about why the "command-line" is "bullshittery" in the first place he compares Python to Excel and declares Excel superior because it can make graphs much more easily. His students apparently ask why use Python when you can use Excel.

First off, Python is not a command line. Secondly, what the fuck does he think Excel's source code is written in? MS Publisher? They're just different things.

The author does not seem to understand why command line tools still work the way they worked in the sixties (as he loves to point out), which is because it just works. He mentions git, the version management system that uses the command line mostly, but forgets to mention that although git could surely have (and has) a GUI, it would disable you from doing many things. I often pipe git output to other tools, something you just cannot do with a GUI.

Oh, and he uses nano for command line text editing. I suppose that also explains a thing or two when working with Vim or Emacs is so much better than any other text editor.

Kalium13• 12 years ago

In fact, git has multiple GUIs. Often a user will use multiple in their git workflow, because git is too complex to fit neatly into one UI.

aet3• 12 years ago

Is it possible to make Git as easy as Dropbox?

Dewie2• 12 years ago

Python to Excel and declares Excel superior because it can make graphs much more easily.

I don't really think he meant to say that. I think he is using a writing style which emphasizes with different view points, namely seasoned programmers and their befuddlement with normal user's inefficient tools, and the normal users' befuddlement with the seasoned programmers seemingly arcane tools that seemingly accomplish the same things that their prettier, more easy to use tools already do.

loup-vaillant8• 12 years ago

[he] declares Excel superior because it can make graphs much more easily.

You should read more carefully. He claimed no such thing. His real claim was that users tend to react like that. And we programmers must be prepared for such backlash.

The author does not seem to understand why command line tools still work the way they worked in the sixties

The author is saying the users don't understand. He knows about their goodness. The problem is to teach it.

He mentions git, the version management system that uses the command line mostly, but forgets to mention that although git could surely have (and has) a GUI, it would disable you from doing many things.

Not necessarily: http://tonsky.me/blog/reinventing-git-interface/

I often pipe git output to other tools, something you just cannot do with a GUI.

A nice GUI could, among other things, record what you just did and translate it into the command line equivalent. But more importantly, when you have a GUI, the command line no longer have to act as an interactive user interface. It can now be dedicated to what it does best: API.

Removing the command like would bring the apocalypse (as it mostly did on Windows), but merely adding a GUI can make things much better.

Oh, and he uses nano for command line text editing. I suppose that also explains a thing or two when working with Vim or Emacs is so much better than any other text editor.

He knows about the Vim/Emacs holy flame wars, so he most likely used one or the other. But go teach either of them to the hapless student. The Emacs tutorial alone takes several dozens minutes to get through. Extremely useful, but quite steep. Nano on the other hand can be taught in seconds. So it is more likely the first text editor the students will see. If you want them to write Python code and you only have an hour before they leave, you have little choice.

Kalium13• 12 years ago

Clearly, all of the people who make useful and powerful tools should spend less time on useful features and more time on shiny Apple-style UIs. Because who wants useful features when you can make software cater to people who don't want to learn?

This author's problem is not with bullshit. It's that he doesn't believe he should have to understand his tools at all. He wants them to magically work because he knows what he wants them to do.

Do you want to do novel, interesting things with your tools? Don't expect someone else to have paved the way for you first.

hvs• 12 years ago

While I can sympathize, does anyone else feel it is odd that we are talking about a Computer Science professor complaining about how basic unix tools work? Sure, they aren't perfect, but I think it is going a bit far to say that CS researchers shouldn't have to understand on the other 99.999% of developers in the world use computers. This feels a bit ivory tower to me.

mcguire10• 12 years ago

Consider that a computer science professor beginning a research program with students is roughly equivalent to a software development manager, just recently promoted from a development role.

fidotron2• 12 years ago

I think it's got to be time to revisit some of the ideas of the Smalltalk or Lisp machines, principally that of one language all the way down from app to lowest level.

One of the biggest pieces of brain damage the UNIXes have inflicted on the world is the division between command environment and programming language. It may have once served a purpose, but in modern systems we have a level of madness that is the equivalent of expecting a literature student to master all Indo-European languages before being able to work in French.

The growing importance of services over the network and containers gives some cause for optimism, since if we can agree about the interfaces between services (yeah, big if) then each container can hopefully become a steadily simpler world unto itself.

pyre4• 12 years ago

Yes. We should definitely require everyone to use the One True Way! Because that has always worked so well in the past!

[P.S. You also fail to explain why not just use Microsoft for the One True Way rather than a LISP machine. Microsoft is all about making their tools work together with other Microsoft tools so that everyone should do things the Microsoft Way!]

sanderjd• 12 years ago

I think there is really a market for this sort of thing now or in the near future. Most people now have a computer in their pocket capable of doing all their personal computing, but incapable of doing most of their professional computing. So there's a big opening for purpose-built professional systems, and in that arena, I think Smalltalk style system malleability could be a big win.

rlpb• 12 years ago

He mentions "bullshit" 19 times in the article. It's a shame he isn't able to describe why it is bullshit (apart from that he has to use it) or how we could make it any better.

The command line is an incredibly powerful tool, which allows us to very quickly combine components together in many more ways than use cases that could be considered by a GUI developer. This is why he's having to use a command line in the first place: no GUI developer has addressed his very specific use case, and nobody has figured out a better of doing these things in the general case than the command line.

It would be great if it were easy to learn. But short of any real workable suggestions, nothing is going to change.

In general, I understand the word "bullshit" to mean something that has been constructed to obstruct somebody in a way that isn't actually necessary. I don't see how that applies here. The command line is the only known way for him to achieve his goals. By definition, then, it cannot be bullshit.

mrbill2• 12 years ago

This results in people like a friend of mine, who has a Masters degree in computer science... yet I had to walk her through swapping out the power supply in her PC.

CocaKoala2• 12 years ago

What should having a master's degree in computer science indicate about somebody's ability to swap out a power supply? I'm curious.

mrbill2• 12 years ago

I would think that someone who would go so far as to get a masters degree in a topic would also be curious enough to know exactly how some of the tools used in that field plug together and work? Am I wrong here?

avz2• 12 years ago

Everybody's knowledge breaks down if you dig deep enough. You can for example ask how to produce the metal for the wiring of that power supply and from which minerals and ores. You don't need to know this since this isn't computer science and other people provide the materials for you. So the issue boils down to two questions. A philosophical one and a practical one.

Does understanding of the power supply belong to computer science or to some other discipline, like electrical engineering?

Do you need to swap your power supply on your own or can you rely on other people offering this as a service? Apple's move to prevent users from replacing batteries in iPhones offers an interesting perspective here.

Sirenos• 12 years ago

Except computer science isn't about hardware.

aet3• 12 years ago

"A computer scientist specializes in the theory of computation and the design of computational systems." Computer science is about hardware e.g. VLSI. It is a shame that CS undergrads aren't forced to learn a little more about circuits, etc

chrisprobert• 12 years ago

I also enjoyed his post on the "Two Cultures of Computing": http://pgbovine.net/two-cultures-of-computing.htm

Definitely something worth thinking about as we build tools for other developers/engineers to use.

ghshephard• 12 years ago

Every domain - whether it be physics, chemistry, electrical systems, biology, and yes, even computing systems, has an initial training window in which the practitioner requires some training on the the basic elements of the toolset. Initially, it's painful, and annoying, as you are literally starting from the ground, and you have no understanding of even the language to understand the instructions you are being given. But, eventually you develop a vocabulary, and learn the rudiments of the tools - even if they are completely peripheral to your actual domain of research.

I have a friend who wanted to be a cook - she was interested in food, when she went to chef school for a couple years, and she was learning how to cook meat, they spent two-three days just talking about knives. Apparently knives are very important to chefs. So they taught her about the metals, the edges, the sharpening, the care and maintenance - the entire science of knives. She went in with no expectations she was going to have to learn any of this stuff - but before she could start cooking, she had to learn how to use her tools, and jigs, and moran edges, and Scandinavian Grind, and what the hell does this have to do with cooking again?

Same with working in a field that requires you use software systems. The command-line is your toolset. Master it, and you can do anything. Don't master it, and you are like a chef who is using whatever knives happen to be in the kitchen, and in whatever condition they happen to be in.

But, if he thinks that we have "command-line bullshittery" today, when 95% of the time I can just type "brew install foo", or "apt-get update bar", and have a pretty damn good chance that 95% of my dependencies are fullfilled, and if not, a quick google of the error message will usually give me a stack-exchange/server-fault/stack-overflow solution, I can't imagine what he thought working with these systems was like in the 1990s...

pyre4• 12 years ago

Maybe he's pining for a development environment that looks like this: http://fc05.deviantart.net/images/i/2003/2/5/0/Operation_Swo...

Then his students will know that it's on the bleeding-edge of The Future.

efaref• 12 years ago

A computer science professor complaining about command-line "bullshittery" is like an astrophysicist complaining about telescope "bullshittery". C'mon, it's the 21st century, we should just be able to take a rocket to other planets!

mike_ivanov• 12 years ago

Some of his students went into debt to receive this sort of "education". How sad.

minimax• 12 years ago

git pipe fork pipe stdout pipe stderr apt-get revert rollback pipe pipe grep pipe to less make install redirect rm rm ls ls -l ls tar -zxvf rm rm rm ssh mv ssh curl wget pip

Anyone care to speculate why he is using curl for one thing and wget for something else?

jackmaney3• 12 years ago

Not to mention that even if one replaces "pipe" with "|", the resulting string would return a syntax error on the shell? When the hell would you pipe the command "git" to anything (not "git log", not "git diff", but "git")?

bostik• 12 years ago

This sounds like a problem that should be easily solved with a simple metapackage, installable either via apt-get or zypper.

Add the university repo, install metapackage and have dependency resolution take care of the rest.

tdicola2• 12 years ago

Yep that's what I was thinking. If he's annoyed that students are wasting too much time setting up software, they should hire some folks to build a nice repository of packages and maintain them.

bjornlouser• 12 years ago

when did command line builds of free software that enable one to " ... be 10x or even 100x more productive than peers who don't know how to set up those tools." become too tedious and esoteric for computer science students to master without hand holding?

fla• 12 years ago

Dear Professor,

Computing might help solving your problem. [1]

[1]https://www.virtualbox.org/

dkhenry• 12 years ago

posts like this are why 90% of students coming out of these university's can't pass simple programming assessments.

lazyant• 12 years ago

Welcome to real life outside academia, where you need to step down from the ivory tower and actually get your hands dirty. Incidentally we never had automatic configuration and deployment as easy as now with all the recent tools.

tdicola2• 12 years ago

I've been on both sides of the fence, I remember being a frustrated student who couldn't get libraries, tools, etc. to work, and now I have some years of experience and know the ropes of working in a Linux environment. My best advice for people struggling is that if something feels difficult and painful, you're probably doing it wrong and should look for an easier way.

I remember I would bang my head against a wall trying to get Linux/Unix software to compile in Windows using cygwin, mingw, etc. and it was just a complete nightmare. However running a Linux virtual machine on Windows is drop dead easy--you can have one up in seconds with something like Vagrant. Much, much easier to get stuff working in a real Linux environment.

Once you're in a modern Linux environment if you're compiling software yourself, again you're probably doing it wrong. Check out your package manager and there's probably a compiled version already available. Is it too old? Look for someone else who's compiled it, or an experimental package source and try that next. Still can't find it? Ok download the source and give it a shot. If you have to do anything more than a typical './configure && make && sudo make install' step back and see if maybe you're doing something wrong.

putzdown• 12 years ago

So here's my theory. Whenever we have the experience of a field of useful knowledge being "bullshittery," it's because we ourselves learned it in an ad-hoc and unsystematic way. There's no question that the C language, say, is complex and full of arcana, both syntactic and from the library. But most people go about learning it with the expectation that this is so. We're often accompanied by friends and guides. Learning a language we usually have the sense that there's a fairly clear beginning, middle, and (just maybe) end--or at least saturation point, where our mastery is sufficient to do useful work--to our quest.

The (UNIX?) command line feels like bullshittery not, I suggest, because it's intrinsically more arcane and arbitrary than many languages and standard libraries, but because of our procedure of learning. People tend to discover it. We tend to learn each part (command, option) in response to a specific need. And this both compartmentalizes our knowledge (we know what "ls -la" does but not what "ls -a" does) and makes us quick forgetters (on the idea that what we learn for a specific purpose we tend not to retain for long).

Therefore, don't blame the command line. Blame the educational process. And if you want to experience the command line without frustration--not as "bullshittery" but as simply a tool you understand and have mastered--the solution is to do what the author seems to allude to, which is to take a more systematic and deliberate approach to learning the thing.

ribs• 12 years ago

Why doesn't he set up a config management server, like puppet or chef or ansible, and have students get set up that way? Or provision some VMs? Or provide some Docker containers?

I have tweeted him thusly.

mcguire10• 12 years ago

Likely because student A's research requires tools X, Y, and Z, while student B's research requires X, L, and M.

trarman• 12 years ago

I have the opposite problem. I hate GUI bullshittery. With the command-line I can string together multiple tools with options to work the way I need them to.

If you throw in a tool with nothing but a fancy GUI, I can't simply pipe data through it. I have to find buttons to click, search for hidden menus, resize windows, etc. What a waste of time! Give me a man page any day!

He says Excel is superior because it makes charts easier? Guy is in the wrong field. Learn some octave or matlab.

jackmaney3• 12 years ago

Agreed. I find any of R, Python (with scikit-learn, pandas, etc), or even SAS far more palatable than SPSS or Excel.

glathull• 12 years ago

Personally, I am sick and tired of all this hammer bullshittery. I just want to build houses. Now I have to hold this nail in one hand and hit it with the heavy thing??!! WTF is this all about? What if I hit my thumb? It's bullshit is what it is. I mean, come on. This is stone age technology. You want me to start with a hammer and nails? I want the damn nail gun now! I don't care if I don't know what to do with it. I'm an entitled brat.

More seriously, every real program is going to interact with a file system in some way at some point. You should learn how to use that before you nail gun yourself in the thumb. Unfortunately that means learning some command line "bullshittery."

recursive3• 12 years ago

I think you missed the analogy. Hitting a nail with a hammer is directly contributing toward the task the builder is trying to accomplish. It is intrinsic, not incidental.

mnarayan01• 12 years ago

From (quickly) reading the article, it seems like "drudgery" would be a much more apropos word choice. Though given that he doesn't appear to describe how to do so, maybe not.

Where I agree with him is that there can be a tendency to make command line usage into something of a fetish; computer science should not be a command line competition. On the other hand, making effective use of command line tools will eventually help almost anyone doing computer scienciey things immensely.

In summary (and possibly what the article was trying to say), students shouldn't feel embarrassed or intimidated by a complex series of command line invocations; eventually they may know what they all mean, but they're just a tool.

robotresearcher2• 12 years ago

Folks, this prof isn't saying the command line isn't worth knowing. He's saying that learning it costs time which would be better spent on actual research. Becoming the three-millionth sed expert does not expand CS.

The command line is the best available technology for many tasks. We're going to need these skills for the foreseeable future and maybe forever. CS PhD students are trainees and its OK if they invest in learning it. But for the prof, it's completely reasonable and rational to want to minimize the total time his group spends on this stuff, for productivity's sake.

I'm a CS prof and I spend way too much time wrestling with flaky build scripts, incorrect docs, overly-specific dependencies, etc, because the original author did not put in the extra effort to make the code usable by a third party. This apparent economy costs a huge amount of other people's time.

Bearing this stuff in mind, a prof can actively try to minimize this overhead by (i) decent design, (ii) a bit of spit and polish before releasing code, and (iii) maintaining lab environments that already work (VM/container images, etc.). Let's train the students to care about this stuff.

edit: To clarify, I'm not saying that the author of some open source code owes me anything in terms of usability or my time. But if they want to maximize their impact, a little effort in that direction can pay off. Like writing a paper that is easy to follow rather than an ugly brain-dump.

Roboprog4• 12 years ago

(iii) sounds on the mark, and perhaps what the guy really wants to say is that there is a market to help profs do that prebuilt setup???

Fuzzwah• 12 years ago

There's a market, but no money.

gear54rus• 12 years ago

From my (not-so-big) experience in IT, the sub-fields of this field are really tightly networked together (e.g. if you understand piping in bash, you can then learn about stdin and stdout concepts in your future C program, or the other way around). Therefore I really can't see how could one try to excel in one area and not touch the others.

While you are learning about the area of your choice, you inevitably understand ideas that hold true for many other areas (provided you possess the perception to see them).

Expanding on these ideas, in turn, may help you understand something new about your area of choice again. Not willing to do so feels like a wasted opportunity.

wffurr• 12 years ago

Clearly a lot of HN readers have Unix command-line Stockholm syndrome.

It's a sad fact that having facility with the Unix command line enables one to be more productive, but that's because because it gives you access to a huge array of useful tools, not due to any innate qualities of the interface.

It's a clear case of worse-is-better.

lukev• 12 years ago

I really want to agree with you - its easy to look at the command line and point out how it's suboptimal.

Unfortunately, there's a dearth of real, better alternatives. There's a lot of ideas, and a lot of toy projects, but there's nothing that's actually viable to which we can migrate.

Except, maybe the Microsoft dev stack. But I still prefer the *nix suite.

mcguire10• 12 years ago

I'm going to quote the majority of an entire section of Guo's post because it (a) demonstrates what I believe to be the core of his problem, and (b) is incredibly offensive to me as a computing professional:

"Here is a common productivity bottleneck faced by students working on applied computer science research:

"1. Advisor and student discuss high-level research ideas by doodling on the whiteboard. Awesomeness ensues.

"2. Student leaves advisor's office feeling pumped and knowing exactly what they need to do to implement those ideas in code.

"3. Student tries to get started on programming but immediately gets stuck since they don't know how to handle all of the command-line bullshittery required to set up their coding environment with the proper libraries, tools, and frameworks.

"Many students get discouraged and turned off from research when they hit the wall in step 3.

"There is a huge disconnect between the elegant high-level ideas discussed on the whiteboard (while presumably sipping cappuccinos) and the grimy, grungy, terrible command-line bullshittery required to set up a computing environment suitable for implementing those ideas in code. This gulf of execution is tremendously frustrating for highly-capable and motivated students who just didn't happen to spend 10,000 hours of their youth wrestling with nasty command-line interfaces."

If I were to try to write a summary of this complaint, I believe I would have to boil it down to, "Useful work is indeed often difficult."

I'm certain the HN community is familiar with the concept that ideas are a dime a dozen, that execution is indeed the important part. I'm also certain a significant chunk of the readership has also spent "10,000 hours of their youth wrestling with nasty command-line interfaces", by which I mean "has learned the fundamentals of how computers work" and "has spent some time understanding how to use their tools."

So along comes Philip Guo, who has, in his career, developed the ability to convert whiteboard doodles to working prototypes (I'll withhold my rant about academic quality code for today). Now, however, he's in the position of every developer promoted to management: he has whiteboard doodles that he would like his loyal minions to convert to functioning code and his minions are having difficulties. And, like almost every other recent promotee, he blames the tools involved. Because, after all, "this bullshit is not intellectually interesting in any way", at least to him; he's only interested in the whiteboard awesomeness, after all.

Programming is difficult. Guo knows this. The tools could be better. (Are there significantly better tools out there than "command-line bullshit"? Not that I'm aware of. Maybe the Plan 9 stuff, Acme, etc., but that comes with its own stack of bullshit and a reduced environment to boot.) But is actually learning to program really incidental complexity? Is it really "bullshit"?

pessimizer2• 12 years ago

It isn't offensive to me in any way as a computing professional. It's exactly what happened to me in college.

"Programming is difficult. Guo knows this. The tools could be better. (Are there significantly better tools out there than 'command-line bullshit'? Not that I'm aware of. Maybe the Plan 9 stuff, Acme, etc., but that comes with its own stack of bullshit and a reduced environment to boot.) But is actually learning to program really incidental complexity? Is it really 'bullshit'?"

Command-line bullshit has little to nothing to do with programming.

Rusky• 12 years ago

The problem is not that programming or "useful work" is difficult. The problem is that getting your environment to the point of being able to do "useful work" is more complex than it needs to be.

The problem is exactly that, as you say, "the tools could be better." The various parts of the system don't work together well enough. The installation process requires understanding systems that shouldn't even need to exist.

porqupine• 12 years ago

Despite the amount of criticism on here (and while I agree the article is a bit over the top), I do think I am strongly partial to this sentiment nevertheless.

I think part of the problem is that amongst coders (a much greater population than that of people doing real research in computer science) fluency in a number of arcane syntaxes is perceived as something 'cool', finally it's a standard that's been ingrained for years and years.

But the amount of say 'command-line-tool' or 'library' specific syntax, that must be re-learned for a new tool (which you might use once in a month, or a library you might use for one function, is very often a giant time sink). It's not cool. It's not intellectually interesting to memorize syntax.

msluyter• 12 years ago

+1 for introducing me to the term "bullshittery."

My undergrad was in music and when I went for a master's in CS, I knew like, nothing, and suffered under a lot of the command line bullshittery he refers to. And most profs at the master's level were unwilling/unable to help with that sort of thing. One of the most useful classes I took was a (totally non-degree fulfilling) undergrad course in Unix system administration.

Nonetheless, I believe that mastering the command line and general Unix environment is still rather essential. Things like basic vim usage, or readline shortcuts (http://www.bigsmoke.us/readline/shortcuts). I still see experienced folks cursoring around on the command line a character at a time, which sort of makes me cringe.

It might be interesting to see some sort of catch-all "systems skills" CS course that covers a lot of this sort of thing.

aeze• 12 years ago

The article actually irritated me a little bit.

zwerdlds2• 12 years ago

Sounds like a docker/vagrant use case.

I do disagree with the thesis though. If youre building on a tool you ought to learn how to use it properly. Otherwise what happens when something breaks? Placing "sensible defaults" works until you need to change the tools functionality, at which point, if you dont know how to use the tool you will be once again lost.

It's a whack-a-mole method vs the dynamite approach IMHO.

pessimizer2• 12 years ago

Learning how to deal with the unix command line is awful, and unix is awful, but it's also the best thing we have right now, and using it is a skill which will pay off again and again over decades.

Helping people to learn it is definitely torture, but it's honorable work that contributes to society in a big way.

I have no understanding of the point of view of the angry comments on this thread, although I suspect that they're based on the headline. The author clearly thinks that the command line is the greatest thing since sliced bread. From the article:

"What is wonderful about doing applied computer science research in the modern era is that there are thousands of pieces of free software and other computer-based tools that researchers can leverage to create their research software. With the right set of tools, one can be 10x or even 100x more productive than peers who don't know how to set up those tools."

"[...]perhaps what is more important to a researcher than programming ability is adeptness at dealing with command-line bullshittery, since that enables one to become 10x or even 100x more productive than peers by finding, installing, configuring, customizing, and remixing the appropriate pieces of free software."

"As an advisor, I've found that one of the highest-leverage activities that I do with my students is guiding them through the intricacies of command-line bullshittery. There is simply no substitute for sitting down with them one-on-one on their laptop and walking them through all of the arcane commands to type, what they each mean, and how to interpret the bullshit output that's barfed out to the drab terminal."

agilebyte• 12 years ago

The people developing command line tools become proficient quickly, writing and improving on existing CLI apps. If they were to spend time writing GUIs, we wouldn't have all these wonderful CLI apps to begin with.

If enough people are complaining about a CLI app, then maybe there is a niche for a company to explore and write a GUI no? Until then, the students need to invest some of their time (rather than money) to use a (usually) open source software.

bkeroack• 12 years ago

Perhaps he should try to explain the rich history of this "command-line bullshittery". It was the original OOP, before anyone articulated it. Encapsulating functionality into small, easily understandable programs that could be chained together was an amazing breakthrough in software engineering at the time. If you look at old Unix sources, the source code for many utilities was only a few hundred lines at most. Essentially the size of what we would consider a "class" today.

Whenever I'm getting frustrated with this stuff I try to step back and realize that we're all standing on the shoulders of giants. It would not be possible to do all this grand whiteboarding were it not for all the work done before us to make it possible. We'd still be grappling with how to wire together transistors and magnetic core memory.

super_mario2• 12 years ago

Command line interface is also the most powerful and at times the most usable interface (in a lot of cases you simply can not do certain things in the GUI, and UI that prevents you from doing something is by definition less usable).

Given the choice between GUI and command line to achieve a task I would always choose command line way. Command line takes a while to understand and use effectively. So yes there is initial cost to reach certain proficiency, but then new world of possibilities opens and you become freakishly effective and powerful in expressing yourself and telling computer what to do, so much so that resorting to GUI (clicking on pictures) becomes hopelessly frustrating most of the time.

There is a hierarchy of skills needed to be effective in the CLI:

1. Learn to touch type at decent speed

2. Learn advanced text editor (really this comes down to Vim/Emacs, either one of them will serve your for the rest of your life).

3. Learn general purpose tools to do basic filesystem things (manipulate files, traverse directories etc).

4. Learn general purpose tools to transform structured text to arbitrary "shape". Things like sed, awk are considered advanced here, but simpler things like cut, tr, colrm, paste etc are often powerful enough and sufficient to achieve the task.

5. Learn to search for information. Learn find command, learn grep (ack/ag), learn regular expressions (amplifying the power of your sed/awk scripts).

6. Learn more specialized tools for specific problems

7. Learn a scripting language of your choice and stick with it. Python, Perl or Ruby are great choices but some are better suited for certain tasks or areas of specialization.

Note that any UNIX distro shipped in the last 10 years has all of these tools pre-installed. All you need is to spend some time practicing. Some things like touch typing and advanced editor may take longer time because of muscle memory involved, but basics of simpler tools can be learned in a few minutes to hours.

And of course while doing any of this, you will be interacting with your shell, so learn its basic and some advanced features.

After all this, you are really a master of your data. Finding, extracting, slicing, presenting data etc become trivial chores.

kjetil• 12 years ago

With love to all the UNIX apologists in this thread: you're missing the point.

Why isn't software easier to install and configure? Why are we still using relics from the 70s as the base of our systems? Why are we so focused on mastering the tools of the past instead of creating new tools for the future?

I'm a long time Linux and BSD user. I sit in a shell for hours every day. And I still can't believe why we put up with this crap.

Shell scripts and one-liners. Ls and cat and grep and uniq and sed and awk and tar and loads of other low-level tools that you haphazardly string together. For the love of all that is holy, just use a real programming language!

With the powerful computers of today, we still interface to other computers through a text-based bottleneck that's basically from the stone age. What's up with that?

OK, this turned out to be a little rant. Don't take it the wrong way; I love UNIX, but I also hate all the cruft that just seems to accumulate and never gets cleaned up. I think we could do a lot better.

super_mario2• 12 years ago

shell is a programming language. You are interactively programming and in a more efficient way too. You are telling the computer to do things for you (instead of clicking on pictures) or speaking in more verbose language.

So I don't really get your objection. Even if you programmed what you wanted computer to do in some other "real" language you would be doing exactly the same thing only in different syntax. And source code would still be even bigger bottleneck :D. Try writing python script to wget a file and extract it to local file system. I think you will find it's more time consuming :D.

dllthomas10• 12 years ago

A linguistics professor I was speaking with once put it (paraphrasing from memory): "With a shell, you have all the expressibility of a language. With pointy-clicky, you're reduced to pointing at things and grunting."

cheald3• 12 years ago

Why isn't software easier to install and configure?

Because nearly always, one size does not fit all. Those four thousand config options are there because at some point, someone needed each of them. Software is complex - a one-click install-and-run is fine for consumer apps built for known hardware running in tightly controlled environments, but once you get out of that sandbox, there's a lot to account for.

We're using "relics", but they're relics that have been battle-tested and continuously improved over the years. We use them because they work. Where redundancy is found, it's often wrapped up into abstractions (see: Makefiles, package managers) that enable operation at a higher level. I think it's myopic to look at computing and accuse it of being some archaic system that has remained untouched for 40 years.

I think we could do a lot better.

I wholeheartedly welcome your impending attempt.

Kalium13• 12 years ago

With love to all the UNIX apologists in this thread: you're missing the point.

We didn't miss the point. We just don't agree with it. We use text-based systems where we assemble small things into the tools we need because one shot do-exactly-what-I-need items are impossible to anticipate.

We know these needs cannot be anticipated. Instead of trying and failing, we do one better. We create an array of tools that can be assembled into the ones we need at any given moment. The result is the power and expressiveness to be flexible and suit the needs of the moment.

Imagine if your car could only ever go to places the manufacturer cared about when it was made. Probably not what you want.

I think we could do a lot better.

We await your thoughts with bated breath.

aroberge• 12 years ago

As someone who has spent a fair bit of his life teaching, I find it "fascinating" to see the reactions of the users here, most of which appear to have at least some experience with programming and none (from what I can see) having any significant teaching experience to draw upon.

Teaching beginners can be extremely challenging. A majority of beginning students, no matter what field is being taught to them, will simply not master the material and will easily get "tripped" by something that advanced users will find trivial.

Most of what P. Guo wrote on his various articles, including this one, strongly resonnates with me, and jives with my own experience as a teacher. His Online Python Tutor tool [1] is a fantastic teaching (and learning!) aid and shows his understanding of what is needed to help beginners learn.

[1] http://pythontutor.com/

juanre2• 12 years ago

I completely agree. The only beginners I have taught were my sons ---I wrote what I thought were the basics for them in a book, The Hackers Ways [1]--- but P. Guo's articles very much resonate with me as well. Helping them see that the tools were the means to an end, something they could master and that would make them much more efficient, was a big part of the effort.

[1] http://juanreyero.com/hacker-ways/index.html

mcguire10• 12 years ago

1. Are graduate computer science students starting their research, not taking classes) really "beginners"?

2. Would Gou consider learning Python to be "intellectually stimulating" or "bullishit", considering that his stated goal is to "produce publications in an applied computer science field".

fiatjaf3• 12 years ago

These students are clearly not interested in computers, and they should be dropped by the university to go work on some other field.

tim333• 12 years ago

I'm reminded of Massimo di Pierro, who teaches comp sci as his day job, talking about the problems of getting students using that stuff http://www.youtube.com/watch?v=iMUX9NdN8YE&feature=youtu.be&... which led him to write web2py which helps avoid some command line stuff for web dev though you'd still probably need it after a while.

ntrepid8• 12 years ago

Behold a man standing on the shoulders of giants and complaining that the giants are not tall enough.

juanre2• 12 years ago

I've often had the impression that mastery of command line tools is seen by some as a rite of passage. Looking at many of the comments elicited by this article certainly makes me think so.

I don't think he is trying to belittle command line tools: he clearly sees them as a pre-requisite for his line of work that is not fulfilled by many of his students. His use of "bullshit" may be unfortunate, but I think he's only trying to ---correctly--- point out to his students that the tools are the just that: tools that should not be intimidating and that will help them be much more efficient.

sehugg• 12 years ago

After his students finish their papers or projects or whatever, most of them will go and stick their code into a .tgz file on a website, unpackaged, unmaintained and forgotten about until years later when another researcher needs to refer to it, who will then grumble and complain about command-line bullshittery...

naunga• 12 years ago

I wonder if he has a colleague in the English department who wrote an article complaining about the amount of "notebook bullshitery" that an English student has to go through to start writing an essay.

Calling out the pains that students go through having to go to a store and pick out a notebook? And the woes and confusion that they experience with faced with the vast and baffling array of colors, paper sizes, page counts, line widths, and so on.

I found the woes outlined in this article to be dubious. This is simply a "first world problem".

On the other hand, I did get a warm-fuzzy feeling regarding my job security, since I am more than comfortable at the command line and further: learning to use tools I have never encountered before.

recursive3• 12 years ago

There is a lot less bullshittery surrounding the acquisition of notebooks and pens than there is around setting up certain software. A ~year ago I decided to set up a rails development environment, having no experience with it or ruby. I assembled a list of repeatable instructions as I went. It took me about two days, and had over 50 steps. If you are experienced with the domain, you could probably do it in 5 minutes, but there is not a good reason why it should require so much specialized knowledge just to get it set up.

kylebgorman3• 12 years ago

I am a jr. CS professor working in an applied field (computational linguistics). I have found myself repeatedly giving the same impromptu lectures to RAs and graduate students concerning various UNIX tools. This is a clear inefficiency (I could have planned the lectures ahead of time, with nice illustrative examples, and given it just once). The scarier thing (for me) was that I occasionally observe students (as well as many open-source contributors) doing something in an enormously complex way because they have not yet _discovered_ tools like `tee` or `cut` (it is hard to know just how much time is wasted in this fashion).

Consequently, a colleague and I created a course entitled Research Programming, which is taken by all our masters students, as well as some RAs and Ph.D. students. At first, some of my other colleagues were skeptical; they felt that students should already know this material (true in a normative sense, perhaps, but contrary to fact). However, the "economy" argument appears to have won them over and the class is going well thus far.

Personally, I don't agree with Guo's assertion that command-line skullduggery is intellectually unstimulating. But it's unfortunate when it gets between you and scientific (or business) progress, and as an educator I hope to minimize that.

drostie2• 12 years ago

That's awesome.

The only thing I think would make it even better would be if we could make it an interactive course online, since so much of these details about shells, commands, etc. are things which happen in a computer context anyway. If several of the major could collaborate on a common research-programming tutorial and release it openly, it would be a huge win for more fields than just computational linguistics.

I've seen some related links to put shells in the browser, so it's more than just an academic speculation; I think one could put the site together with under a thousand hours of programmer time at this point. Hosting costs might be a concern, but application development is probably not too difficult.

kylebgorman3• 12 years ago

Good idea, I'd love to enable online learning for this type of material. In the meantime, I'm putting handouts for the first unit ("UNIX") online, and the second unit ("Python") will use iPython notebooks. There may be Coursera (etc.) offerings that cover similar material, but I haven't looked into it yet.

dubya2• 12 years ago

This sounds like a gap in the curriculum that could be filled by grad students. I'm not in CS, but I know that the bioinformatics students have organized a course that covers some basics of the unix command line tools, probably because data conversion is a big part of most projects.

There's also "Best practices for scientific computing" (http://arxiv.org/abs/1210.0530) that covers some of the same ground, and physics departments usually have a scientific computing course as well.

Also, if it's really a one-off use of something, the department IT person could probably do it for you.

kylebgorman3• 12 years ago

The curriculum goes far beyond the knowledge of most IT professionals (not to mention our current graduate students), as it is focused on research; IT pros don't normally know much about, say, Python libraries for machine learning or signal processing.

bb0wn2• 12 years ago

There is no 'overcoming' of command-line bullshittery -- it's a barrier to entry and computer professionals should be glad that it's non-trivial to become productive with your environment -- the time that you put into becoming fluent in POSIX and all the requisite pays off 10x. It makes you that much more valuable to have put in the time to learn an interface that is powerful.

It's hard because there is a lot you can do with it, both in research and in the workplace. I am really biased against software people who are not good with their tools. I don't want to work with people that are counting on their fingers and toes trying to get stuff done -- I want to work with people who embrace the tools and take pride in using them well. Is playing the scales keyboard bullshittery, or the fundamental skill required of being able to play beautiful music?

robotresearcher2• 12 years ago

It's one thing to be proud of building an accomplished skillset. But it's another to be glad that other people don't have access to that power.

bb0wn2• 12 years ago

I'm not glad that others don't have it -- I'm just glad that it's non-trivial because it means that learning it is an accomplishment worthy of taking pride in. As far as it being a barrier to entry is concerned, I think that having to learn something that is not-so-simple to become a professional separates the wheat from the chaff in the same way that someone like a professional exam or certification would. I have worked with newbs in the past who don't know how to use their tools correctly, and they're usually a net negative re: productivity on a project.

mbish• 12 years ago

This seems to me akin to saying that: because learning how to use a tool is never research (the tool is already built) it is therefore bullshit and a waste of time. I like the command line so I'm biased in this direction but you need to know your tools in order to do work. What if a mechanic didn't want to deal with all that "how-to-use-a-wrench" bullshit and just wanted to fix things!

mverwijs• 12 years ago

I honestly don't see the problem. Don't want your students to learn the innards of computers running Linux? Configure a Vagrant instance and have your students Vagrant-Up. Could have been done inside the time he took to write that blogpost.

cheald3• 12 years ago

It sounds like maybe he should invest some research time into improving software interfaces.

I get the frustration with getting mired in infrastructure before you can actually start producing useful things. I experience that regularly. But you know what? It's going to be there for your whole career, and it's often a useful driver which inspires people to improve the things that annoy them. I usually learn very useful things in the process which improve my ability to comprehend the entire system.

I think there's a balance to be found here - it's unreasonable to expect students to spend their entire semister in an advanced CS research class learning how to use rudimentary tools, but maybe it's also unreasonable to put people into advanced CS research classes before they've learned how to use their field's rudimentary tools. The student should know how to do it, and the professor should attempt to ensure that they don't have to.

On a different topic: My anecdotal experience is that academically produced or oriented software is nearly always among the worst offenders of this - it is overly complex, poorly documented, and not well supported. This leads me to wonder if there are attitudes in academia that such things are "bullshit" that get in the way of "real work", which lead to this problem in the first place.

jordanpg2• 12 years ago

Isn't this just blossoming into the vi/emacs/IDE debate couched in different language?

Can you -- in principle -- accomplish goal X with tool Y? Yes, almost always. Some people will be far, far more efficient with the CLI and likewise some people will do better with GUIs.

There are some tasks that are only amenable to CLI work, like kernel development. And there are some that are only amenable to enormous IDEs like mobile app development.

The author is saying that for the scope of his problems, the CLI just gets in the way. This isn't always true. He is careful not to make any sweeping generalizations.

Roboprog4• 12 years ago

Tangent - but why does mobile app development require a huge IDE??? Aside from how bad the APIs are.

What if mobile app APIs had a simple interface to add layouts to views, and widgets with their callbacks (in one call) to the layouts? And what if all the widgets and layouts had so much of their behaviour standardized in the base classes that you almost never needed to look up, auto-complete or otherwise remember 100 distinct classes' use cases???

jordanpg2• 12 years ago

I'm most familiar with Android, but I would say it's because there is too much infrastructure under the hood.

Google seems interested in abstracting away almost everything about the framework and build process into the Gradle plugin in Android Studio, to create the illusion of what you're describing. But behind the scenes, all of the machinery if IntelliJ is making that happen.

Related question I asked a while back on SO: http://stackoverflow.com/questions/24151396/how-can-i-view-a...

msamwald• 12 years ago

I think that the article describes a sentiment that far more people dealing with computer science / programming share than usually perceived. I think it is mostly because for the (seemingly warranted) fear of being called out as unprofessional by other people in the field.

I'll come out of the closet: I also wonder why Bash is still so important in 2014. I think it simply is unintuitive and cumbersome for all tasks that you don't routinely do, and only a small minority of people that are using Bash are doing it in such a fluid way that the potential benefits of the command line can be reaped. For the rest of us it mainly feels like http://xkcd.com/1168/

In summary, I think that: 1) Bash is overripe for being replaced by something else, perhaps a more user-friendly and interactive shell that is built from the ground up and not a fruitsalad of ad-hoc solutions created a few decades ago. 2) Using Bash and praising its benefits is not only so popular because it has positive practical aspects. It also seems to have become culturally ingrained in the geek community in an irrational way, like an initiation rite that separates insiders from outsiders.

neilellis• 12 years ago

As someone who lives eats and breathes the Bash shell, I can say quite confidently that I can do most things I need just using this 'bulshittery'. Ignorance in my book is not a virtue.

What I failed to understand is exactly what was so difficult about writing and compiling software? This process can be as easy or as difficult as you wish.

Do they not have IDEs, surely that would help. I use IntelliJ as an IDE and it pretty much does everything for me, including finding dependencies.

Maybe I'm missing something, but if I can build complex Java programs without reaching for the command line surely this can be done in a scripting language without any complexity.

Also I would suggest looking at the dockerized development environments out there. (e.g. http://awvessel.github.io/)

Maybe I need to re-read this and see what I missed.

0xdeadbeefbabe2• 12 years ago

It takes a tremendous amount of command-line bullshittery to install, set up, and configure all of this wonderful free software.

Then it isn't wonderful (or even free in some ways). Has the author considered what Kay might mean when he called linux a budget of bad ideas? I thought academia had more freedom to reject pragmatic stuff when it doesn't suit their needs.

The author might be able to help his students best by reviving an interest in systems research.

dllthomas10• 12 years ago

Porting my comment from yesterday's posting of this (https://news.ycombinator.com/item?id=8432955):

"I strive to remove incidental complexity for my students, so that they can focus on the intrinsic complexity of their research."

A worthy goal. I think, though, that calling out the fact that it's command line serves to distract more than illuminate - at least the reader and possibly the author. There are doubtless unnecessary steps between idea and code. However, shoving those in a graphical wizard doesn't make it any better.

Roboprog4• 12 years ago

Cry me a river.

I've yet to meet the person who was a programming god in trendy-language-of-the-day, but couldn't use a command line (perhaps with a few weeks' practice to change shells / operating systems). Maybe it's more to do with off-shoring, but whenever I see "programmers" who can only use a GUI, I know I'm in for trouble, and said "programmer" won't actually be good at issuing orders as lines of text for the computer to follow.

Yes, please give me cave paintings and tally marks! Don't make me learn stuff like functions, sets, pipelines or language!

A more valid criticism would have been "some programs/utilities/packages are more complicated than strictly necessary", with examples. Others have pointed out that it should be possible to make a "distribution" of a tool suite (perhaps as simple as a script containing the individual setup commands), rather than making every student run every step by themselves.

Having an "install" clicky thing from a 3rd party that has no options and just works is great. As long as that 3rd party knows all your needs (really knows, not the Apple mind control illusion) and has control of the environment in which you are dropping the package.

Roboprog4• 12 years ago

Wow, this came out kind of bitter. I guess maybe due to too much having to deal with a Windows culture of grotesque ignorance the last few years.

zwerdlds2• 12 years ago

You're right though. Even MS is starting to realize the lack of power in their GUI-centric designers. I will say that, in trying to catch up, theyre way out of practice in buildin powerful/customizable command like tools (see SQLMetal and t4 for starters)

nocman• 12 years ago

I'm a little confused about what the author is really trying to say. In this article, it sounds to me like he is saying that all of this interaction with the command line is arcane, stupid and just needs to go away.

But then I read the other article he wrote which he referenced in this article ("The Two Cultures of Computing" -- http://pgbovine.net/two-cultures-of-computing.htm). In that article he discusses how there are two groups of people -- the "User" group that just wants to do everything with a GUI interface, and the "Programmer" group that likes the intricacies of the command line. There he discusses the difficulty he had with explaining to students the justification for learning the command line tools (which included the fact that they provided more flexibility than the GUI tools alone could achieve).

He concludes the "Two Cultures" article by encouraging folks in the "Programmer" group to bridge the gap between the two cultures:

"You need to gently introduce students to why these tools will eventually make them more productive in the long run, even though there is a steep learning curve at the outset"

Although I dislike the notion that the command line is "bad" or "arcane" just because it has been around for a long time (which the "Two Cultures" article seemed to harp on a lot), I do agree that bridging the cultures is an excellent idea.

And then I go back to the article referenced in this post, which seems to be saying that all this interaction with the command line is just boring, stupid, and has no value. So again, I'm not sure what the author is ultimately trying to say.

Personally, I appreciate both. I love a good GUI that makes common tasks fast as lightning to rip through. But I will always prefer to also have the ability to get down to a lower level, where I can make the tools do even greater things that the author of the GUI did not think of or did not have the time or motiviation to implement.

Also, I think it is a bad idea to discourage students (especially Computer Science students) from learning the nuts and bolts of how computers work. We have enough of a shortage of people with that kind of knowledge already. IMHO we need as many people with those kinds of skills as we can get.

danielweber4• 12 years ago

His thesis isn't "the command line is bullshit."

It's "all that bullshit you have to do (on the command line) is a waste of time and mental energy."

fiatjaf3• 12 years ago

So this guy thinks that the people who build the tools that enable people to "be 10x or even 100x more productive" are bullshitters, while he, the almighty professor who produces a lot of useless "publications" and getting taxpayer money for it, is the nice and important guy on the field of "applied computer science".

hcinfo• 12 years ago

Two things that I've found that help a lot with the problem the author describes: Cloud hosting and YouTube.

We recently created some research software[1] that was somewhat kludgy to set up, and asked a high school student to give it a try. He got stuck on some early steps -- like working with the DVCS.

So we created a YouTube video[2] that walks through provisioning a new host on Digital Ocean, and each step after that. The video got him "un-stuck", and as a fallback he could always abandon what he was doing and simply replicate the entire video step-by-step.

1. https://sharps-ds2.atlassian.net

2. http://youtu.be/p54CCUEhfig

fiatjaf3• 12 years ago

It is strange that nobody commenting in this thread, or in any other thread of HN, will feel the way the author's students feel about the command line.

I would also expect that most people commenting on HN, whatever their degree of knowledge, find the command line and all the things it can do, all the tools they know how to use or they know they can learn, a lot interesting and valuable.

These are the characteristics of people who really love computers and, by consequence, whatever that means, "computer science". The students referred by the the author are not truly interested in the computers as the HN visitors, and they should not be in an university doing "research" with the final objective of producing "publications".

For such a mismatch of interests and people to happen there must be a very misleading set of incentives driving the academia today.

TeMPOraL3• 12 years ago

I feel him. I used to feel in exactly the same way.

This is how you feel when you get overwhelmed by stuff you don't understand, that is completely irrelevant to your goal, but which you are forced to do anyway.

I used to think version control is bullshit back in the days I was first learning to program. I wanted to make games. In particular, I looked for other games made by hobbyists in order to download them, play a bit and then look at the source. It really pissed me off when the only way someone shared their production was through a CVS repository. I wanted a simple .zip download, not some stupid weird Unixy thing (I was running Windows) that was complex, required to download some weird console software (ah, the PATH issues) and was completely irrelevant to my goal of getting up-to-date version of the software. Oh, and they didn't store binaries in CVS so here's another hour wasted for me to try and figure out how to compile that project (fixing dependencies took time).

When NASA released some of their software as CVS repos with no direct download, I got so pissed I vented on a blog (http://temporal.pr0.pl/devblog/2007/06/25/systemy-kontroli-w..., pl_PL only).

At some point I learned SVN though, and then Mercurial and Git, but the main thing that made me stop being annoyed by version control was switching to Linux for primary programming work. Version control feels natural there. GitHub for Windows application also helped.

Another, more recent example. Because of some weird reasons I ended up having to learn how the old RMI in Java worked. Oh, such a pain in the ass it is. I wasted like five to ten hours making it work because of bullshit like XML configurations, setting up servers from (Windows again) command line with magic invocations, weird security issues and dropping manifests in random places. All I wanted to do is to have program A call a function in program B. Things I did to get there were completely irrelevant, but took 90% of time.

Current example - don't ask me to do anything in client-side MVC frameworks. The whole ecosystem around Angular, Ember and Node.js feels like utter bullshit to me. Again, tons of magical invocations, hundreds of megabytes downloaded, hundreds of thousand of files in project directory, all that to achieve what used to take a single alert("Hello world!").

I know, it's probably useful somehow to something. But I wanted to make a simple app, you told me to use Ember, and I wasted half a week and a DVD worth of disk space and I still don't have that app.

So yes, I feel him. It's not that the software or command line sucks. It's that 90% of things he has to do are completely irrelevant to his research, and yet he can't proceed without doing them.

What can we do to solve it? Hide the bullshit. Hide the infrastructure. Just running a program should never require touching version control. Hello world in a web framework should not require setting up test frameworks, downloading half of the Internet and selling your soul to Satan. Make basic use cases simple. Or at least, write tutorials that explain the reasons behind magic invocations and that are targeted to beginners, not people with 10 years of experience in the domain.

Kalium13• 12 years ago

Bear in mind that his use case is that people want to do novel things with their machines. You can simplify, automate, and GUI away things you can anticipate. When someone needs to go beyond that, you can try to give them a helpful framework, but that's it. Quite quickly they're going to be off blazing a trail, armed only with their wits and some basic tools.

But here's the catch - when blazing a trail, you can't expect to find a nicely paved road with regular bus service.

avz2• 12 years ago

I've always been surprised when my friends mention how hard it is to install FOSS packages from scratch. I mean what can be more natural than

    cd directory
    ./configure
    make
    make install

it's almost like in the real life:

    cd kitchen
    ./configure microwave
    make pizza

Isn't it? ;-)

LordIllidan• 12 years ago

Until you're missing an ingredient and need to go all the way to the supermarket. Multiple times.

With a binary, you're just reheating a frozen pizza.

niels_olson• 12 years ago

Coming to programming via biomedical research (I'm the undisputed grand master of microscope bullshittery in my pathology department), I have to say, the wall of command-line bullshittery is much steeper than of anything else. I can count on one hand the people from whom I have had to opportunity to learn anything about the command line in the last 10 years. And I can count the number of hours of that learning I got too.

I submit a more appropriate analogy might be mechanical engineering. You want to be a mechanical engineer and learn to build bridges. You'll start in middle school shop class and learn about sand paper and how to draw isometric projections. By the time you graduate high school, you can spot a 1/4" x 20 thread pitch (aka "quarter-twenty") from 10 feet. You'll learn the relationship between shims and wire gauges, drill sizes, flutes, the geometry of volutes and involutes, the merits of various steels, the dangers of overhead welding, the properties of concretes, how to patch carbon fiber defects, the grades and types of ball bearings, needle bearings, babbitt bearings, and their lubricants, crush plates, etc, etc. Along the way you'll have to learn the difference between Y and Delta three-phase power, how to analyze an op-amp, and how to model how the center of gravity of a car with shocks changes when it goes around a corner.

The difference is that we have a shop program in middle schools and high schools that cultivate that culture. There is precious little in terms of UNIX culture cultivation before college.

russelluresti• 12 years ago

On one hand, I get where he's coming from. You'll get more people interested in something when you give them direct access to all the cool stuff first and none of the setup. This is how Khan Academy structured their Programming courses. If people don't have immediate roadblocks thrown in their way, they'll want to stick around longer.

On the other hand, there is a common mindset in computer science and programming of "I don't understand this, it must be bullshit." This article comes fairly close to fitting this mindset, and may actually exist squarely within those parameters.

Ultimately, everything you want to do is going to come with prerequisites. If you want to go out in public, you must first learn to dress yourself. If you want to cook something, you must learn how to properly heat your oven, stove, grill, or whatever. You can't just say "I'm making chicken - the appropriate cookware should already be set up for me and heated to whatever is necessary to make chicken." There are so many different configurations you can use to create software or cook chicken that you can't just say "this is the one setup that will take care of all your chicken-cooking needs."

There's always been a strange fear of the command-line interface. Maybe because it looks old we think it's automatically shit. "This is the way people did stuff in the 70s - how have we not improved on this yet?" Just because a method has been around for a long time doesn't automatically mean it's out-dated. In fact, if a method HAS survived that long, it usually means it's got something going for it.

Maybe, in the future, someone will come up with software that's better to use to get you started on something new. However, the person who creates that software is going to have to have a fundamental understanding of how those command-line tools actually work. If you want to improve something, you need to understand it and not just dismiss it as "bullshit". And if you're a mentor in the CS field, maybe just complaining and expecting someone ELSE to fix the CS field-specific issues isn't the approach you should be taking.

chubot• 12 years ago

It's true that the "command line" takes a surprisingly long to learn. I've definitely surpassed the 10,000 hour mark, and I'm STILL learning things. (People say that about Vim alone :) )

That's probably because it all superficially looks the same (sequences of characters), but there's are a lot of depth and functionality with many different concepts -- e.g. compilers (cross-compilers too), data compression, networking, storage and file systems, hardware devices, crypto, parallelism, data formats for every problem domain, several important little languages, several syntaxes for the same language (regexes, etc.).

But I think you basically have to learn it to become computer literate, and all computer science researchers should be computer literate.

Otherwise you can't "fend for yourself". If you are relying on your professor to set shit up for you, that's just lame and you will not be useful in many other contexts.

What's the alternative? I think it's "mouse bullshittery", which is even worse. For example, compiling software stuff on Unix can be hard. But I used to use Windows, and it's 10x harder. I remember the last experience I really had with Windows was installing Visual Studio to compile Python, and I hit some trivial but showstopping but in the CDs, and gave up in disgust. The workaround was to copy some DLL somewhere in your \system32 folder, and I'm like "really?"

I don't really use Macs for development either, because it's annoying to set up compilers. (It's 10x less annoying than Windows, but 10x more annoying than Linux).

vicaya• 12 years ago

He should help his students understand how to acquire basic skills in a field, which is a prerequisite skill to do any research.

It start with a web search these days.

There is simply no excuse, when the knowledge of these basic-skill/bullshittery is openly available.

You know, teach a guy how to fish...

soyiuz• 12 years ago

I don't like the reasoning in the author's argument at all. He is begging the question: it is "bullshittery because it is bullshit." By assuming the command line as not an intellectually rich environment, he creates an intellectually impoverished and frustrating experience for his students.

The command line is far from that. It is an incredibly standardized environment, elegant compared with "black box" gui-driven cluttered all-in-one solutions (looking at you Stata) for programming or doing data science. It encourages code that "does one thing well" and which produces input/output streams in an unencumbered, plain text format.

These ideas may not be interesting for the development of new search algorithms, but they are incredibly interesting for the developing of a new search engine, for example, or a new word processor (as interfaces between human and machine). Using the command line well develops into the daily discipline of economy, simplicity, and common sense. If command line is bullshit, what does he think is not?

Helping my students overcome command-line bullshittery

Related Stories