The need for tooling and the need for mastering your tools

http://muratbuffalo.blogspot.com/2018/03/master-your-tools.html

by r4um • 8 years ago

222 68 8 years ago

muratbuffalo.blogspot.com

jitl• 8 years ago

I like tools. I like building them, and I like using good ones. I love the feeling of leverage from using my tools together to accomplish in days things that would have taken weeks or months without.

As a tools author though I struggle to produce things that are both powerful yet also easy to use. I tend to build things that are too “UNIX-y” - and I end up with blank stares from co-workers (users) when I ask them, “did you read the —-help output?” Or likewise when I suggest that perhaps they can wrap tools in bash scripts or aliases and bolt things together...

jacobolus8• 8 years ago

You are working with a broader toolbox than your coworkers. It's like if I ask a carpenter how to attach two boards together and they send me some instructions for making a fancy blind dovetail joint... doesn't do me any good because I don't know the basics or own the necessary basic tools. I would be better off with some nails and a hammer.

But that doesn't mean your tool is bad or too 'hard to use'.

Const-me• 8 years ago

I struggle to produce things that are both powerful yet also easy to use.

The problem is really hard. I can build such tools, but it takes me 2-4 times longer than making an equally powerful tool with rudimentary UX barely enough for me only.

Also I’m from Windows world, here “easy to use” == “rich GUI”. Also shell scripts + text streams is rarely a good way to compose stuff here, people are used to APIs/libraries/COM interfaces.

TeMPOraL4• 8 years ago

There are two ways to see this. One is, your tool is too difficult to use to get the 1x value from it. The other is, your coworkers lack the necessary knowledge to extract 10x value from it.

I know that stuff shouldn't be complex for complexity's sake, but I also frequently notice an anti-intellectual attitude in our profession. I see people learning just about enough of programming to land a job, and then refusing to learn any further. This is a pretty weird behaviour, IMO, but sadly not uncommon.

shostack• 8 years ago

I'm a somewhat technical marketer who enjoys building lots of fun tools in spreadsheets for a chunk of what I do.

While the raw functionality is often very complex and met with blank stares, I've realized I find new excitement in refining and building pretty "front-ends" in the spreadsheet that simplify things for end-users.

Finding interest in the challenges of simplifying and polishing for others is one path forward with your struggle. It was a definite shift as before my focus was on making them as powerful as possible. Only when I realized that the most powerful tool in the world is useless unless people adopt it was I able to break free of my old mental model.

spc4762• 8 years ago

About 20 years ago a friend and I were doing some contract work for a company. I was an undergrad (Comp Sci) and she was a graduate student (also Comp Sci---same college). Our task was to add links to a glossary in about 100 pages of HTML and my friend just started doing it all by hand (well, editing each file one at a time). I had to fight with her to stop and just think about the problem. We were both familiar with Unix, and we were working on a Unix system (SCO, but the company had blown thousands of dollars for the entire SCO works) and I realized we had access to lex.

Normally, one uses lex in conjunction with compilers, but it would work for this job---as each word was found, generate the output with a link to the glossary; otherwise just copy the input to the output. Half an hour later we were done with the entire task.

I just now realized that there may have been more to the approaches than I realized. I was looking at this very tedious task and not wanting to do it. Why not have the computer do the boring grunt work of finding and editing? But we were being paid hourly and it could have been that my friend was looking forward to doing this very tedious job for hours and making that much more money. I am reminded of the Upton Sinclair quote, "It is difficult to get a man to understand something, when his salary depends upon his not understanding it."

It could also have been that I think differently and realized the application of a standard Unix tool normally used for one task was appropriate for this task and my friend just couldn't see it. Either could be possible.

I have to wonder how much of this is due to "not understanding that the computer can be instructed to do the boring stuff" versus the "I don't want to code myself out of a job."

jacobolus8• 8 years ago

Finding and deploying tools you know about (but wouldn't immediately think of) in non-standard ways is a meta-cognitive skill which can be practiced. It's possible you had some relevant past problem-solving experience which your friend did not. It is also possible that you had systematically been socialized to pursue different high-level strategies toward work – I have many friends who were children of working professionals and were taught to value 'working hard' above looking for shortcuts.

Aside: It's not only the explicit prospect of hours of work that might have stymied your friend... research suggests that when working for a reward people are typically less creative and less thoughtful than when intrinsically motivated (see Alfie Kohn's book Punished by Rewards).

golergka2• 8 years ago

But we were being paid hourly

As someone who occasionally hires developers, hour rates of a grunt who knows how to edit HTML and someone with a CS degree who knows how compilers work and and can dive into lex/yacc during an interview differ significantly.

eecc• 8 years ago

Speaking of tooling and differential mathematics (mentioned in the article itself,) I also basically flunked “Analisi II” (and quite probably never fully understood Analisi I,) the Italian equivalents of Calculus.

Only when I changed Università and retook Analisi II, taught by a different prof, who demonstrated everything using more powerful and meaningful tools did I get it all.

Same thing with electromagnetism; couldn’t ever not make a mistake with those pesky Maxwell equations until I bumped into some lecture notes treating them with Diadic Calculus (a kind of tensor) and voila, I passed.

My brain is pretty average, particularly when referring to working memory. Give me a strong abstraction and I get it, ask me to wade through the minutiae and I lose it.

castle-bravo• 8 years ago

I don't think you're alone in this. A lot of my coursework is presented as a series of exercises in memorizing solutions to special cases and applying those solutions in a treadmill of quizzes, assignments, and exams. I find myself craving better, more powerful abstractions that subsume the special cases; complex numbers are a great source of satisfaction to me for this reason.

jacobolus8• 8 years ago

This is the main reason I am ambivalent about the typical school (even the best ones I know about, and from first grade up through the undergraduate level). Compared to the amount of time spent, relatively little is focused on mastering all of the metacognitive skills required for expert problem solving in any domain. Almost all work is done at the shallowest surface level with shallow delayed feedback and an overemphasis on punishing minor errors or non-standard output formatting, and on judging/ranking the students. There is a parade of 'content' to get through, but little chance for students to work on problems which are truly difficult (for the student at their current level, e.g. taking more than a few hours to figure out a single problem) or larger-scale projects of personal interest. Very seldom is the work revisited or iterated. Ideas and methods from one discipline are seldom applied to another. Students are systematically discouraged from taking any kind of risk. (This is not primarily the fault of individual educators: scaling close personal attention from experts is just uneconomical given current societal resources available for education, and schools have many difficult constraints on pedagogy.)

phreack• 8 years ago

A unique approach to scaling personal attention I've seen work wonders is having a good teacher choose several well-performing students per course to voluntarily tutor the next generation on course-wide group projects. In very few years there were about ten capable tutors, each personally and closely helping just one or two 4-5 person groups on large assignments -- overall grades, student satisfaction and projects' quality skyrocketed.

barrkel• 8 years ago

It's almost as if school was designed to create labour for factory work.

Although this is interesting: http://hackeducation.com/2015/04/25/factory-model

triangleman• 8 years ago

There is a parade of 'content' to get through

Did you know that the content is also shallow? According to E.D. Hirsch [1], one of the reasons low-income and other marginalized groups do bad in schools after elementary school is because they lack the exposure to "cultural vocabulary" and end up feeling like a fish out of water when any more challenging material comes up.

So in line with the spirit of the OP, some of the "tooling" that we need in education is simply a deeper understanding of the national culture that all well-educated people have naturally adopted.

For instance, you could help low income students by simply having them watch shows like The Big Bang Theory and then maybe have a discussion about it in class. (And I say this as someone who doesn't like that show. But I know it speaks a very strong cultural language used by well-educated Americans.)

[1] https://en.wikipedia.org/wiki/Cultural_literacy

npsimons• 8 years ago

For me, a big turning point, in learning anything really, came when I started looking at the bibliography/references. Any text can be considered "introductory", and I believe all classes are. If you really want to go deeper, you have to go to the references and study outside class, and no one tells you this.

jacobolus8• 8 years ago

I don’t think watching TV sitcoms is going to cut it. If you look at the density of ideas, richness of references, complexity of sentences, etc., pretty much any written material (novels, nonfiction aimed at teenagers, magazines, newspapers, ...) blows those out of the water. The teacher would do better to just read to the students for 30m every day, instead of having them watch a show for that time.

red0point• 8 years ago

I study at ETH Zurich in Switzerland. In each one of my courses all students devote countless hours to solving the problems the instructors throw at us, there is explicit assistance given if you need it and each problem you solve is then corrected & graded (if you want).

It‘s very satisfying, say in an algorithms class, to come up with an algorithm that satisfies the given runtime requirements, often taking ideas from multiple basis courses - and this is expected. It‘s always explicitly stated that the goal is not to, say, remember an algorithm like Dijkstra by heart, but to internalize how to come up with such an algorithm, as the exam is also structured around coming up with new algorithms.

s3nnyy• 8 years ago

I doubt you have to come up with Djikstra-like algorithms on your own, do you? (The Dijkstra algorithm is kind of groundbreaking and famous.) Applying Dijkstra is actually what we had to do once in one exam at TU Munich. It was the "Sechser Bremse" task though.

taeric3• 8 years ago

Mastery of tooling is easily the most successful nerdsnipe you can do in an org. It abounds with metaphors to sharp physical tools that are ironically required to perform tasks safely. And, to an extent, it is obvious to see that people with more mastery over their toolchest will be able to outperform those of lesser mastery.

However, it is also easy to find evidence that quality of tooling does not translate to quality products in software. Not the least piece of evidence is Linux and the rise of free as in beer tooling.

I think it can easily diverge into navel gazing. It isn't that tooling is unimportant. But that it should not become the focus. If you want to make quality products, obsess over the quality of your product. Don't get distracted by what you know about what went into it. Focus on the product. Period.

Yes, you will at times have to settle for a proxy measure. No, they will not be perfect. Nor can they be. Don't fixate on a single metric, but collect many and constantly ask yourself what the metric is telling you about the product. Not about itself.

jacobolus8• 8 years ago

This is mostly talking about a different type of tooling and a different type of mastery. 'Tools' here might be something like 'immutable data structures', 'formal analysis of algorithms', 'linear algebra', 'debugging', 'usability studies', or 'project management', rather than some specific piece of software.

The examples given (paxos and map-reduce) are very high level concepts, which open the door to products which would be impossible or qualitatively very different without them.

taeric3• 8 years ago

To an extent, sure. But, I could replace "experience" with every usage of "tool" in the article and find that it makes more sense to me.

When you lack experience building anything, it seems natural to think that the tools used by practitioners enable them to do what they do. After gaining experience, lots of it, you find that the tools can help build a scaffolding, but what you are really building can often be obscured by the scaffolding. Which is why it is not uncommon to see some experts that can work with minimal tooling. They have built their ability to see the product and end goal, without the need of some tooling.

Which is not to say you can always get by without tooling. No amount of internalizing the instrument and idea of flying will help a pilot fly without a plane, after all.

accidentalrebel• 8 years ago

The tool for me that I have spent a lot of time learning and improving is my Emacs environment.

It wasn't easy but as soon as I got the hang of things I was able to improve a lot in terms of overall productivity. Knowing how to debug when things go wrong and where to look to solve it was the first step in my learning journey. I eventually started heavily customizing by making scripts and packages to further suit my needs.

After you master a tool, you internalize it and go minimalist, and reduce it to first principles. This line made me feel like I'm on the right track. Just recently I've cleared up the fluff in my emacs config leaving only simple but effective and essential tools. I realized that I've made a lot of edits to my config and that each one taught me something that eventually led to where i am now. In a way, clearing things out was a learning experience in itself.

I still have a lot to learn though. Just recently I've made a jump to reading the enacs source and even took the time to contribute to Remacs, a Rust port of Emacs. I'll admit that this one is a bit overwhelming and may even seem overdoing it, but it is still a fun learning experience.

I sure do love my tool.

jrimbault2• 8 years ago

Same with vim (and other text editors), at first I started customizing heavily. Now I can get going with just 3 settings.

    syntax on
    set number
    set relativenumber

weaksauce3• 8 years ago

As a long time vim user and lover of the idea of vim. emacs is better at being a vim than vim is. (org, magit, helm, swoop, and all the other great packages are a bonus and work as well as they do because of the way emacs was created.) vim modal editing is life but vim's implementation is really bad comparatively.

flukus2• 8 years ago

I've invested a great deal of time getting plugins to work, scripting my own, customizing every aspect of vim but I really wish I listened to the people saying all along that "vim is the text editor, unix is the IDE". Since I started following this advice I haven't looked back.

The two main reasons I ignored this advice initially is that I was mainly on windows and that vim has some great windowing abilities. I knew about cygwin as an answer to the first but I didn't know about tmux previously, perhaps I didn't need to know because vim already had a similar feature set.

Nowadays my vim setup is almost as minimal as yours.

yosamino• 8 years ago

set relativenumber

why thank you!

liveoneggs• 8 years ago

I wanted to add the few that I like but I think they're all defaults now :)

set ruler set wildmenu set showcmd set scrolloff=4

DanielBMarkham• 8 years ago

I've always had a dream of starting a programming school. Educating new programmers seems like something that's not done very well. (Or maybe I have very limited experience here.)

If I ever did, I think I'd start with text processing. If you are an expert at slicing, dicing, processing, and piping text? A bunch of other stuff suddenly becomes much easier.

cbm-vic-20• 8 years ago

I have a friend who teaches woodworking to kids; for their first project, he teaches them how to build a wooden mallet.

ccvannorman• 8 years ago

in Bay Area there are many programming / development / game development schools. I also worked on CodeHero.org for a time.

What specifically do you think is wrong with the existing approaches? It's an area I also care a lot about and would like to gain more insight into. :-]

z3t4• 8 years ago

start a blog series.

TeMPOraL4• 8 years ago

I keep refactoring my Emacs config, but it still grows instead of shrinking.

Maybe I'm not "there" yet. But for now, I feel that the more things I start doing in Emacs, the more productive I get (and the more fun I'm having). They say Emacs is an OS; I find it to be a damn great one. Its UI model is a better fit for 90% of things I do on a computer than the standard one.

ninjakeyboard• 8 years ago

+1 - this article makes me think of my journey into emacs

joesmo• 8 years ago

Almost every software tool decision I make is one of garbage quality with features vs ok quality with no features. I can choose a text editor that does the basics but nothing else I need or an ide that does everything poorly. From browsers to package managers to debuggers to operations tools, the quality of software tools is absolutely atrocious. Why master any of this garbage when more garbage tools will be required later? It's hard to stay sane just using them. Yes, non software tools like reading and writing comprehension are useful, but for the most part, investing heavily in a tool that's bound to fail over and over again seems like a terrible idea. Then again, there simply isn't enough time to write proper versions of these tools. But amazing things have been built with much less tooling than we use today which leads me to think that the proliferation of software tools partly exists only to make the tool developers happy as they create useless rube Goldberg machines (especially in js land).

montyf• 8 years ago

I wrote a blog post on tool mastery a few days ago, mostly in response to an egregious article named "You are not your tools"[0], but also to write down some of my recent thoughts on this topic[1].

I agree with the OP that internalizing a tool changes your brain for the better. Everyone knows that learning a new language (human language) expands your brain and improves your ability to learn more languages. Let's continue this example, just to make things clearer. Mastering foreign languages is not a cakewalk, despite scattered marketing claims that you can do it in a year or even three months. From the OP:

Just having a passing familiarity with the tool doesn't empower you.

You can't learn Korean just by visiting Korea and waiting for osmosis to take effect. Even people who live in foreign countries for decades fail to learn the language if they don't put in the appropriate effort -- I've seen it happen. But "You are not your tools" makes the following dim-witted assertion:

Spend your time listening and learning from everyone, whatever tools they use: most skills will transfer just fine.

That's exactly the opposite of what you should do. You will internalize nothing with this plan, nothing will "transfer" (if anything you will have interference[2]), and you will just end up like so many other mediocre programmers I see. Anyone who's learned a foreign language knows this -- you need skillful Practice and Concentration. Trying to learn two languages at once takes more than twice as much effort than if you focused on one, if you don't do it with great skill.

Quote from my blog post:

There is a huge gap between the average programmer and the real hacker. On one side of the gap is the doe-eyed little boy who goes into every project clueless about how to start, searching for a relevant tutorial on Google while frantically asking questions on StackOverflow. On the other side is the man who plunges into the project with an air of mastery, controlling his environment with a fluid ease which is rare and somewhat dangerous – a man who’s already finishing the task by the time the boy has gotten his first response on StackOverflow. Which side of that gap do you want to be on all your life?

Now returning to the OP.

What qualifies as a tool? Does critical reading qualify? How about writing well?

These absolutely qualify. These are fundamental tools -- mastering them will improve many other areas of your life. But you would not believe how few programmers take the time to improve these skills, or even care. Sometimes I get the feeling that the average programmer hasn't read a single book in his life (apart from maybe some programming or self-improvement books, which are generally poorly written and unimaginative). It shows.

What are the tools that benefited you the most?

Emacs, without question. I took the time to learn it during a spell of happy unemployment as I was living abroad. This one decision has paid enormous dividends in my life, but only because I took the time to truly master this tool. It not only has made me more productive at pretty much everything I do on computers (which in itself is valuable -- the more prolific you are, the better you become), but it made me smarter and more confident on account of having mastered something. Men are supposed to be experts. Mastering something silences your inner critic and cues your brain in to your innate self-worth -- it's how male brains seem to function.

You can learn Vim instead, but please avoid IDEs. Easy-to-use, lowest-common-demoninator, "thoughtless" tools actually seem to be making people dumber. It's the path of least resistance versus the path of blood, sweat and tears -- and they go in opposite directions.

A question the OP should have asked, but didn't:

How do you know when you've mastered a tool?

You will know. It changes the way you see the world. New possibilities arise, the horizon broadens, and you begin to see with lucidity how powerful you are and how far your potential truly goes.

As J. Peterson says,

People create their worlds with the tools they have directly at hand. Faulty tools produce faulty results. Repeated use of the same faulty tools produces the same faulty results.

A quote from Kafka on the Shore:

Pointless thinking is worse than no thinking at all.

[0] https://codewithoutrules.com/2018/03/23/you-are-not-your-too.... These kinds of articles are not just idiotic, but harmful to society -- the attitude expressed here will make people worse at their jobs and at life, and we should speak out against this pernicious mindset.

[1]: I won't post my article here, as my blog can be found on my user page. I posted it on Reddit and received a hostile and, well, juvenile response. A lot of people took issue with the idea that certain tools are better than others (seems like Vim/Emacs are not liked there) and the idea of someone taking mastery seriously ("You must be fun at parties" was, I kid you not, an upvoted response). It's not just about mastering your tools, but mastering your environment by extension. It's about power -- the power to change our world -- for the better. Many people embody a mediocrity mindset and go through life without skill or strategy -- they unconsciously know their own inadequacy, and you can see this on their faces.

[2] https://en.wikipedia.org/wiki/Interference_theory

LandR• 8 years ago

I couldn't agree more with this.

yodsanklai• 8 years ago

You can learn Vim instead, but please avoid IDEs. Easy-to-use, lowest-common-demoninator, "thoughtless" tools actually seem to be making people dumber.

I would argue that you should learn both. It really depends on the context. Vim/emacs are nice, but for certain tasks, IDEs are more suitable. I would also recommend to master your OS shortcuts, we edit a lot of text outside vim/emacs.

flukus2• 8 years ago

I'd begrudgingly agree, but learn vim/Emacs/kak first, along with Unix command line tools. They cripple your understanding of many important things if that's all you know and they impose a lot of limits.

discreteevent• 8 years ago

John Carmack uses an IDE all the time. Didn't seem to affect his productivity or ability to fix things at a very low level.

In general I don't think the editor really makes much difference. The article emphasises tools at a slightly higher level - more algorithmic. I think this is more important.

swaggyBoatswain• 8 years ago

I would argue that learning both is important, but its always better to learn the dumb tools first (IDE). Dumb tools were made for a reason because they are the lowest common denominator and therefore easier to get things done. Your time spent on tooling could have been used elsewhere.

VIM / emacs are great if you work in a unix environment, but as a windows user Id rather have nice GUI tools. Requires less memorization to do the same tasks.

I would also argue against memorizing shortcuts as well, and if you do, only a select few that you often use. Rather have a quick reference file on hand for these shortcuts to memorize.

silveroriole• 8 years ago

“Men are supposed to be experts. Mastering something silences your inner critic and cues your brain in to your innate self-worth -- it's how male brains seem to function.” Women with female brains, on the other hand - er...

OnlyRepliesToBS• 8 years ago

That sentence is describing an author's description of one of the sexes, not attacking the other.

Stop instigating shit.

edwinyzh• 8 years ago

I like software tools which is handy. I needed to do mindmapping in my way and I built MindVisualizer (http://innovationgear.com/mind-mapping-software/); I needed to edit my web pages in my way and I built LIVEditor (http://liveditor.com/index.php).

thriftwy2• 8 years ago

I don't like tuning my tools.

You spend a few hours tinkering with settings, then two years pass like a breeze, you have reinstalled your system / on another employer's laptop, and you have to re-configure everything. From memory. That's frustrating.

Yes, I carry around a .zshrc from the times when I still made the effort.

Otherwise I would prefer convention over configuration. Just provide sane defaults. Please.

Moreover, when you're an infrequent programmer, you would want to invest the least into tuning tools. E.g. I won't waste time setting up configs on a zillion servers that I visit. I'll use bash. It's better today than it was 15 years ago.

mcjiggerlog• 8 years ago

Try setting up a dotfiles git repo. Here's mine - https://github.com/tom-james-watson/dotfiles - it contains everything I need to get productive and, apart from installing a few dependencies, it's a one line install.

majewsky2• 8 years ago

You spend a few hours tinkering with settings, then two years pass like a breeze, you have reinstalled your system / on another employer's laptop, and you have to re-configure everything. From memory. That's frustrating.

That's why we treat configuration as code these days. Put all configuration files that matter in a repo.

And if a tool does not take a plain-text config file, or at least allow you to export settings to a plain-text file for later re-import, it's frankly not worth your time.

thriftwy2• 8 years ago

I am not a DevOps so I won't be able to deploy that repo contents for the fear of my life.

sevagh• 8 years ago

When I was younger, I was into "minimalism" - I would spend hours writing up i3/vim/emacs configs to make things look minimalistic and strip out dependencies e.g. getting things to work without using a DM or DE with some fancy bashrcs, writing my own programs for volume/brightness controls.

Now I minimize my time wasted tweaking things that don't matter. I just use Fedora (stock with Gnome 3) with about 20 combined LOC for vimrc, emacs configs, and bashrcs. I embrace the defaults.

TeMPOraL4• 8 years ago

Just copy configs over. Best, keep a repo with all of them.

Few hours spent on setting up your environment at a new job is still a great bargain if it makes you more productive (and/or makes you enjoy the work more).

camnora• 8 years ago

The book The Art of Learning explains these concepts in depth very well.

The need for tooling and the need for mastering your tools

Related Stories