The problem with slow tools (2021)

https://jeremymikkola.com/posts/2021_04_01_the_problem_with_slow_tools.html

by piinbinary • 3 years ago

62 55 3 years ago

jeremymikkola.com

gumby2• 3 years ago

Kids these days! In my day, compiling took so long we could go get lunch, walking uphill in the snow both ways.

x86x87• 3 years ago

How can you walk uphill BOTH ways?

foundart• 3 years ago

because it's a joke

hyperion2010• 3 years ago

House at top of hill, school at top of hill. Or in this case, lab at top of one hill, lunch at top of another hill.

klardotsh• 3 years ago

Tell me you've never visited Seattle or San Francisco without telling me you've never visited Seattle or San Francisco...

ketralnis• 3 years ago

https://duckduckgo.com/?q=uphill+both+ways

euroderf• 3 years ago

cf. Monty Python

sn41• 3 years ago

... and mind you, there were no fast food restaurants either. So it took a really long time.

musicale2• 3 years ago

Left out the mandatory https://xkcd.com/303/

Now it's "CI tests!"

pjerem2• 3 years ago

I worked in a company where the codebase of a team took hours to compile so it was compiled every nights.

Interestingly, it may have been beneficial to focus because you had to work the entire day without compiling the code. I suppose that in this situation, fast tests is what you need.

martinky24• 3 years ago

Reads like the dude just learned how to multiply and extrapolate and got carried away.

If you spent 2 minutes peeing, and pee 8 times a day, that's 5840 minutes spent peeing a year! Wow! I bet he'd love that one.

cozzyd• 3 years ago

This is what corporate diapers are for.

kmstout• 3 years ago

As with anything in engineering, it depends.

https://www.walmart.com/browse/health/adult-diapers/depend/9...

Dylan16807• 3 years ago

He's doing that in the context of advocating for better tools.

If you could install an update that made you need to pee 10x less often, wouldn't you jump on that opportunity?

Especially if the urge keeps interrupting you when you're right in the middle of solving something?

wredue• 3 years ago

No. I drink lots of water to pee more often, in fact.

It’s a nice reminder to get up and stretch, plus take my eyes off the monitor.

beanjuiceII• 3 years ago

April 1st

jacknews• 3 years ago

Of course every delay is a potential interruption to focus which costs way, way more than the delay itself, so much so it's not worth calculating the actual delay's 'lost time'.

In fact even a momentary delay (my macbook now seems to often stutter for 2-3 seconds switching spaces) is an opportunity for your focus to drift (why is this damn thing so slow, I should get a new machine, actually, I should lookup the latest thinkpad, maybe I can tune memory use in the meantime, how do you tune macos memory, can you even?, etc ...).

nine_k3• 3 years ago

1000 times this.

Also, the time of focus loss grows highly super-linearly with the distracting delay time. One second could be imperceptible, once you get used to it, 15 seconds is enough to get distracted by looking at something else, like checking IM / email and spending 2-3 minutes on returning to the full focus, and 2 minutes is enough to go for a coffee / reddit / YouTube, and a potential loss of an hour of focus time.

josephg• 3 years ago

my macbook now seems to often stutter for 2-3 seconds switching spaces

I get so angry at this sort of thing. The M1 chips are so fast! When they came out they could open basically any program instantly. But now that they’ve been out for awhile, developers everywhere are (of course) just assuming everyone has an M1 or faster and doing even less optimization on their code. So the net effect is that software doesn’t get any faster. We just collectively spent billions of dollars on new hardware with almost no tangible benefit. (The exceptions being high performance computing - like AI or AAA video games where optimizations still matter).

Computers are fast. Slow software comes from lazy developers who steal your cpu cycles instead of doing their jobs properly.

It’s inexcusable.

sanitycheck2• 3 years ago

Cue the usual arguments against performance tuning: "That's premature optimization", "It only has to be fast enough", etc.

It bothers me too. We have astonishing computing power, but outside certain applications we mostly just fritter it away. I would love to use a computer which does everything almost instantly, but instead I have to deal with counting to 2 in my head after pressing WinKey before typing my search term and go-and-make-a-cup-of-tea Gradle build times for pretty trivial projects.

When it comes to Apple though, I think it's somewhat intentional. "Why don't you have an M1/2 yet? Go buy one!"

mwcampbell• 3 years ago

All right, we've been complaining about this forever. How do we fix it? The best thing I can think of, as a developer, is to deliberately use slower computers to test my software. My daily-driver Windows PC is still a quad-core Skylake from 2016, with 16 GB of RAM and a ~500 GB SSD, and I wonder if even that's too much.

anonzzzies• 3 years ago

My 2011 thinkpad is easily many times slower than the m1/m2, yet, when running i3 on my thinkpad, the feel of responsiveness is a lot more snappy as in instance. So in reality, I have a lot less of what you mention on my ancient thinkpad than the m1 for most of what I do daily.

skydhash• 3 years ago

my macbook now seems to often stutter for 2-3 seconds switching spaces

Wow, mine (M1 MBA) is less than one second. I either use command+tab or the trackpad. I’d be crossed too if it took that long for me.

CGamesPlay• 3 years ago

Well, if you do the math, 15 minutes a day times 230 working days a year equals just shy of 60 hours - or 1.5 work weeks - wasted per year. Multiply that by the size of the engineering organization, and this starts to look pretty expensive.

I hate this type of reasoning. 15 minutes a day is 3% of the day. Multiply that by 230 working days a year and it's 3% of the year. Multiply that by the size of the engineering organization, it's 3%. Your maximum savings here is 3%, no matter how many numbers you multiply it by.

cvoss• 3 years ago

All true statements. However, it's conceptually easier to grasp the impact of 3% of my time when I think about what I can or cannot accomplish in a 1.5 week period, rather than a 15 minute period. Some 15 minutes are jam-packed with productivity, while others are a coffee break, so I don't really understand what 15 minutes of my time is worth. Whereas, each work week is nearer to my average productivity, and I have a much clearer idea of what that's worth in terms of work output.

If a person reads this "if you do the math" reasoning and has a different reaction to the same cost depending on how it's presented, then, wasn't it a good thing that the reader was given a variety of ways to think about the cost? (Since, necessarily, the reader's reaction to at least one of the presentations was incorrect.)

nine_k3• 3 years ago

How large is your morgage rate? Imagine that you could shave 3% off its rate, say, turn 7% into 4%. Would it make a difference for you?

Same applies to most companies. Unless they are in an explosive growth phase, or in a catastrophic collapse phase, 3% change in expense / efficiency of the engineering organization is pretty noticeable.

GavinMcG• 3 years ago

Going from 7% to 4% is shaving 43% off your mortgage rate—which of course would make a difference. But it’s not even the same order of magnitude as what we’re talking about here.

coldtea• 3 years ago

Imagine that you could shave 3% off its rate, say, turn 7% into 4%

That's not how percentages work! 7% would turn into ~6.8% if you shave 3% from it.

gumby2• 3 years ago

Companies seem to tolerate the 8% of each Teams or zoom meeting lost on “wait, I can’t get my camera working”, “sorry I’m late; what did I miss?”, etc.

I’m convinced it’s a percent or two of GDP!

jillesvangurp• 3 years ago

Actually you are ignoring the main argument here that people aren't perfect and that the risk of losing more time due to people getting distracted, reading slack, going on long coffee breaks, etc. is substantial and very real. You lose all that time; not just the build time. And having to wait is frustrating and demotivating. And let's face it, a 1 minute build is actually pretty sweet. I've been on teams where it is closer to 20. And worse. That really sucks the life out of you if you have to sit and wait for that to happen fifteen times in a day. People wasting hours per day waiting for builds, CI, etc. to happen is not cheap.

happymellon• 3 years ago

At my current place they have just replaced the build system.

If I want it deployed to the "higher" dev environments I have to watch it for 20 mins so that I can "approve" the override that my base image (that I have no control over) has security risks and I accept them.

If I want to deploy to production, I have to watch and approve the 20 min gatekeeper, them again at 40 mins, and then at 120 mins accept that I want a change ticket.

If I miss any of these then I have to restart the build process. This is an "agile" workplace.

Fuck everyone who claims they agile doesn't work, just stop calling this shit agile.

musicale2• 3 years ago

"Well, let's say you can shave 10 seconds off of the boot time. Multiply that by five million users and thats 50 million seconds, every single day. Over a year, that's probably dozens of lifetimes. So if you make it boot ten seconds faster, you've saved a dozen lives. That's really worth it, don't you think?"

https://www.folklore.org/StoryView.py?story=Saving_Lives.txt

I think of this quote whenever I am waiting for my iPhone to boot. Though I suppose Apple would argue that you don't need to reboot your iPhone every day the way you might have with older computers. There were never a billion Mac users though.

rvba• 3 years ago

That's why I hibernate my computers.

Problem is that Microsoft is trying to kill hibernation (so far they did the first step to hide it, then second step to hide it even better).

Why do the companies not realize that employees coming to office, waiting for computer to start, then opening their programs - is an incredible waste of time and productivity? Why companies dont push microsoft to make hibernation the default? (plus some smart feature for updates)

Folcon• 3 years ago

This trend is so incredibly frustrating and will likely be the final straw that makes me go back to linux on my desktop instead of windows.

I can put up with many things, but not this.

The machine already reboots for goodness knows what reason. Probably some update that someone decided was far more critical than my machine being up.

Also why can't updates run on windows without a reboot? We're getting to the point where other than design, windows feels like the volunteer run OS and that's not me saying the linux desktop design is bad by any measure, just that I prefer windows.

vintagedave• 3 years ago

What does Windows promote instead of hibernation? Ie what -- if anything -- is what the OS or users are supposed to use?

phodge• 3 years ago

I ran a survey and some one-on-one interviews of 30 or so engineers at my workplace to get some real data about the impact of slow tools. IIRC, the results were something like this:

* Most engineers can maintain focus on a task for up to 10 seconds waiting for a slow tool. However a couple of engineers (myself included) will get derailed at only about 3-5 seconds.

* A tool that runs for more than a minute will cause just about every engineer to switch task while it is completing. (This means they stop working on their highest-impact work and likely spend time on lower-impact things).

* A tool that runs for more than 15 minutes will make any engineer have to start deliberate multi-tasking where they are trying to work on two major work items at once.

Also, I wouldn't have read the article or posted this comment if it weren't for the slow CI pipelines at my workplace.

lloeki• 3 years ago

One of the worst unspoken bits is the cognitive load associated with such long running things, e.g compile or CI, which ideally are successful but that's not always the case, so you're doing something else while - consciously or not - anxiously waiting for the other thing to complete.

And when - when! - that job fails, you can be pretty sure it racks up some cortisol as one has to interrupt† your second task, pop back to the original context, curse at oneself for forgetting that comma or whatever simple thing, rerun, rinse, and repeat.

† The alternative is to finish (or reach a checkpoint) your other task, but that has cumulative latency effects on the duration of that other task that is supposed to be your main one.

onion2k2• 3 years ago

If everyone on the team is easily distracted then why would you focus on speeding up slow tooling instead of focusing on making your team resilient to distractions?

TheCoelacanth• 3 years ago

Ah, yes, just change human nature. That sounds much easier than speeding up a program.

revskill• 3 years ago

Slow tool is annoying, but it's not critical.

Bad code, shitty micromanagement, garbage meetings is the main issues.

smitty1e• 3 years ago

The assertion here is that these lost bits of time are purely negative.

I come back with fresh eyes and spot things, get coffee, tidy the desk, make a call, stretch the leg, use the restroom, &c.

But I don't pass myself off as an ur-coder, so possibly I don't count.

pacifika• 3 years ago

These articles about extrapolating the value of a minute of wasted time into a years worth of value, always forget to divide them by the average productive hours a day. I assume you’re not optimising for zero breaks.

reportgunner• 3 years ago

The problem with multiplying all the way to the organization size is that it's always going to end up being a big number.

Imagine all the oxygen we would save if we all stopped saying "bless you" every time someone sneezes, multiplied by the population of the world, multiplied by an average time per year someone sneezes multipled by an average lifetime multipled by an average amount of people in the room who can hear the sneeze.

wisienkas• 3 years ago

I'm not going to say that slow tools is not an issue, however I think slow management processes has a far greater cost to an organization, and the amount of people in management or business analysts doing psedou-work. Which the author also indicate end of the post, how many of us don't have a few 30-60 minutes meetings a week that effectively have no value to anyone.

sshine• 3 years ago

Compiling and deploying effectively destroys my train of thought every time.

The only thing that has helped is to reduce build times from minutes to seconds.

By disabling link-time optimization in pre-production and by throwing out Docker because it doesn't play nice with build tool caches.

pictur• 3 years ago

Applications that try to do too much in one application often start to experience serious performance problems as time goes on. Almost all applications written with node.js have this problem.

tpoacher• 3 years ago

Good article. It focuses on build times as the bottelneck, but actually for me most of the time that bottleneck is 2FA.

By the time I've managed to sign in to my email to double check an instruction, I've already forgotten what I was looking for...

The problem with slow tools (2021)

Related Stories