Stone knives and bear skins – there is no money in tools

https://queue.acm.org/detail.cfm?id=3606027

by sizzle • 3 years ago

69 65 3 years ago

queue.acm.org

ggm4• 3 years ago

There no money in tools: The Dutch company ASML would like to talk to you.

The article does actually say that in highly specialised niches tool making is fantastically lucrative. So, don't mistake the headline for ground truth.

Also there is good reason to believe tool making since flint knapping has been high status, and demands surplus production to liberate skilled labour to making tools which have immensely high use value: tools are magnifiers of human productivity and are worth a lot.

The point is that when commodities are cheap, and tools like gdb can be considered a commodity, the relative advantage of "a better one" has to be weighed up with using a screwdriver as a hammer. Thus printf() debugging.

JohnFen4• 3 years ago

Well said. I know there's money in tools because I've been making money making and selling tools for a long time.

But not every kind of tool. A gdb replacement would be a hard sell unless it managed to actually perform miracles. But specialty tools of the sort that can save a relatively small group of devs major pain and/or time? There's good money in that for a small business. That the market is too small to be attractive to large companies is just gravy.

sanxiyn2• 3 years ago

Time-travel seems miraculous to me, but uptake seems very slow to me.

I hope https://www.replay.io/ succeeds.

coldtea7• 3 years ago

Well said. I know there's money in tools because I've been making money making and selling tools for a long time

Couldn't a succesful actor say the same? "I know there's money in acting, because I've been making money acting for a long time".

And yet we'd instantly understand that doesn't refute the sense the statement was meant, and that acting isn't really the field you wanna pursue if you want to make money".

Same for any other niche endeavor, where there are thousands more lucrative fields and niches, and much fewer succesful examples compared to other areas.

rhelz2• 3 years ago

All a gdb replacement has to do to succeed is not require the binary to be codesigned.

funcDropShadow• 3 years ago

What kind of tools do you build? I'd be interested in your opinion on what are the preconditions for tools to profitable.

fmajid• 3 years ago

IDA Pro competes with gdb and Ghidra and seems profitable enough:

https://hex-rays.com/IDA-pro/

Then you have the people who sold shovels and pickaxes to prospectors during the Gold Rush

368700592• 3 years ago

ASML's crown jewels are not mere "tools", they are machines. It is bleeding edge technology and entirely irrelevant to a general discussion of economics of 'tool makers'.

https://www.youtube.com/watch?v=5Ge2RcvDlgw

ggm4• 3 years ago

and GDB isn't a machine?

nwiswell• 3 years ago

In the industry they are very literally called "tools". No one calls them "machines". E.g., "the fab is ordering another six EUV tools".

Similarly a lathe or vertical mill is "a machine tool". They're ideally suited for a very specific task in the process of making a thing. It's possible for something to be both a machine and a tool.

If we're being pedantic, the ASML tools (and indeed any modern wafer fabrication equipment) are also robots.

More related to the business of tool making broadly, however, the entire industry of semiconductor wafer fabrication equipment (WFE) is quite profitable and margins tend to be generous. Much of the industry is very technical and does involve "bleeding edge technology", but not all of it. The publicly traded Japanese company Disco makes tools for wafer grinding[1], which are not really any more complex than your typical computer-controlled metal cutting machine tools, yet they are quite profitable[2]. Just look at those headline margin figures!

It seems to be a matter of niche and competition.

[1] https://www.disco.co.jp/eg/products/grinder/dfg8560.html

[2] https://www.disco.co.jp/eg/ir/library/doc/film/20230420.pdf

trollerator23• 3 years ago

In software tools. Hardware tools are hardware.

PostOnce2• 3 years ago

There are lots of profitable software tool companies, aren't there? Atlassian, JetBrains, Microsoft, most of them are smaller than this but still "money"; I live in open-source land, but everyone else I know works somewhere that spends money on all kinds of tools.

It's just that the fruit in software tools doesn't hang so low, I think.

evrimoztamur• 3 years ago

If your tool is an integral part of a larger chain, as is the case for ASML's machines being in the chain of silicon chips, that's a different story than a tool which simply eases the process by a small factor.

UV lithography is the only way to make advanced chips required for various other tools and appliances, but an OS debugger (as the author puts it) is just a nicer printf.

There's big money in one, and small money in the other.

coldtea7• 3 years ago

There no money in tools: The Dutch company ASML would like to talk to you.

"You made a general argument. Here's a single counter-example, on a very tiny niche, compared to thousands of examples in your favor, making much more money in other areas of IT".

Dalewyn2• 3 years ago

I mean, you're using a computer to shitpost here. Everything in that computer, let alone the computer as a whole, is a tool that made its manufacturer and any vendors involved lots of money.

PostOnce2• 3 years ago

Here's another multibillion dollar example https://en.wikipedia.org/wiki/Lam_Research

ggm4• 3 years ago

The article specifically reaches into the whole "spanners are too cheap to be worth making" space. The article didn't say "except of course, those gold plated NASA spanners" or "ok: so this is a broad brush but there are notable exceptions"

I called out an exception. I like the article. I think the point is well made. I would say it a bit differently: commodities are very hard to make profitably once the exemplar is good enough, and cheap enough. If you seek higher utility then don't be surprised if the people who use your Tool/Machine/Robot balk at the price, when their goal is met by the lower utility commodity.

If you don't care about lead and tolerance, those fake resistors are just fine for hobbyist use. If your project needs lead free and 0.001% then you really need to pay the premium for originals, not fakes.

And ASML is a niche, but an example of a generic problem in their line of reasoning. For almost any "industry" you care to mention from coathanger bending to lobster fishing, there is somebody making good profit selling what the coathanger-wrangler, or fisherman would call "a tool"

ie: cute argument, but actually... sorry.. niches are everywhere.

sanxiyn2• 3 years ago

Being against a kernel debugger is one of the worst decision Linux ever made. Thankfully those days are behind us.

https://lwn.net/Articles/270089/

Linus Torvalds, in paraphrase:

I happen to believe that stone knives and bear skins force people to think about their problem on a different level.

This is exactly as ridiculous as it sounds.

JohnFen4• 3 years ago

Yeah, I think he overstates the case. I do see people who have become overly dependent on debuggers and that has adversely affected their debugging skills -- but that isn't an indictment of debuggers.

kbenson• 3 years ago

It's ridiculous, but at the simplest level, it's not wrong. Your tools or the lack thereof will change how you think about things, for better or worse depending on the situation.

If your goal is putting venison on the table tonight, vs understanding deer behavior and herd migration and when deer are even in the area in enough numbers and so can even be thought of as a viable food source, a stone knife and a bear skin as tools might promote (or force through necessity) better understanding than a hunting rifle and a large puffy jacket.

That doesn't mean it's "better" or that everyone should be using shitty tools, but it is food for thought, and that perhaps doing it a few times is a learning experience that isn't all negative.

m4632• 3 years ago

kernel debugging on linux sucks (IMHO)

I've tried it and it is really hard to get even the most basic debug output - a symbol'd stack trace of a crash.

If you can't get it at runtime, you need a kernel dump, and trying to get one is a mess. Once you have it, it is hard to match up the dump to the symbols. I believe some of the tools were kernel-version specific and I think when kernel 5 came out they didn't work.

worthless-trash• 3 years ago

I have run "crash" against a live kernel, i have run systemtap against a live kernel, kprobes and written my own C debugging linux kernel issues.

I can't speak for other distros, but you install the relevant debuginfo(symbol files) and then use your tooling, and its been pretty straight forward, at least for the last 10 years.

Hit me up if you ever need assistance doing it on standard x86-64, I'll do what i can to help.

stcroixx• 3 years ago

So not at all ridiculous. I prefer the method of dev Linus follows.

MobiusHorizons2• 3 years ago

How is this ridiculous? I don't think the core of what he is saying (that tools affect how you think) is a matter of opinion. I can certainly see how people would disagree on his conclusion (therefore everyone must use the harder tools).

Tools that force you to hold a model of what's going on in your head and reason about its state require you to think completely differently than tools which hold the state or context for you outside of your head. Think about how different it feels to compose a message in your head, or with a pencil and paper, compared to using a word processor. You can even notice this effect by just typing text with your eyes closed as compared to staring at the screen as you type. The amount of context in your head affects how you think about things. At least for me, the tools I use greatly affect how much context I absorb into my head, and as a result what kind of theory generation and problem solving I can do, especially when a problem is hard enough that it requires several sessions of debugging.

honeybadger1• 3 years ago

While this essay serves as an interesting critique of the current state of operating systems and their development, it perhaps falls prey to the very flaw it seeks to highlight - the tendency to oversimplify the complexities inherent in systems software development and to yearn for a 'perfect' solution that is largely a figment of the author's imagination.

m4632• 3 years ago

I think there are a few things at work.

There is no money in tools because employers (and to some extent employees) don't value them. Also, folks who make tools maybe value them too much? And maybe tools are too easy to copy?

I remember decades ago when purify came out. I tried the demo at work on a project I was working on. Within a few hours, I found an enormous number memory errors. It was enormously helpful. Purify had a high per-seat license cost, and work wouldn't buy it.

Many current employers are a little smarter about things, but I mainly see things like code coverage tools and version control systems, but no real spending on more personal/fashionable tools, even if employees are more efficient.

I think tools that might work better would be those employees bought them themselves - think jetbrains - and priced accordingly. The analogy might be auto mechanics building up their collection from snap-on.

also, warts and all, linux is a nice environment because you can get to the bottom of most things and tools might be less industrial quality, but there are plenty of them.

neilv• 3 years ago

Purify was glorious. As was Saber-C before it.

My employer paid for both. But we sold even more expensive development software ourselves, and bought very expensive workstation and server hardware to run it on, so we were accustomed to the price points.

I like that great software is now open source, and most any kid or adult around the world can run it on a cheap decade-old PC. But I do miss some things about that software development era.

Veserv3• 3 years ago

Employers do not value tools because the employees say the cost-benefit is not worth it. The problem is that the employees do the cost-benefit calculations remembering their time as a poor student when their time was worthless.

When your time is worth $0, a product must be 10x better to be worth paying 10x as much. When your time costs $200,000/yr, a 1% improvement is trivially worth $2,000/yr.

People in software do not internalize this because we can build our own tools. We can solve the problem given time unlike say a chemist who can not build a reaction vessel. When we were poor, we built our own tools and we keep that habit even after we have money and our time is valuable.

It is absolutely ridiculous; the average janitor spends more on tools than a software developer. Truck drivers, secretarys, artists, mechanics, all spend more on tools even though their time is significantly less costly on average. In every other engineering profession companys literally spend $30,000, $100,000, and more a year keeping their high value workers productive.

Only in software do we think that $3,000/yr for Visual Studio, the tool that you literally use the entire time when doing Windows development, or $99 per year to do iOS development are “steep” costs. $99 is like one hour of time; it is ludicrously cheap.

$25,000/yr for a 25% productivity improvement is a no-brainer. The problem is that software people think it is obviously wrong, but the business people think it is obviously right. We do not need to struggle with hand trowels, the smart business people want to give us excavators, we just need to stop telling them no.

Mountain_Skies• 3 years ago

Not sure what it is like for other professions but in software development I've found getting approval for buying tools to be a cumbersome process. The most ridiculous case was a $10 license that took several months to get approved and involved multiple 'C' level executives at a company with about 30k employees. At the same company, I was denied access to a cloud service that the company already had a subscription to because my process would use $0.03 of compute time per month, which we were quite willing to pay. There were no security concerns or anything else, just two mid-level managers trying to extract resources out of each other.

Those are extremes but even simple things like getting a $100 plugin to parse a particular data format ends up being a time-wasting process that far exceeds the cost of the plugin, and if drawn out enough, the savings it would have produced over building it ourselves. As a result, building it ourselves tends to end up being the default choice.

Dalewyn2• 3 years ago

When we were poor, we built our own tools and we keep that habit even after we have money and our time is valuable.

This is a lesson in life that most people really only realize once they start getting old and see more and more people around them die.

Time is money, and as we get older we (hopefully) have more money but we only lose and lose whatever time we have left. Money is theoretically infinite, time is finite. You can borrow money, but you can't borrow time.

At a certain point in life, paying money to save your time (aka use someone else's time) is worth it no matter how expensive it is.

eternityforest• 3 years ago

I don't even think cost benefit is the driving factor. I think making tools is why a lot of devs maintain their interest.

For any given task you'll find 5 commercial solutions that basically make it 1-click, if you're lucky a few FOSS equivalents, and 5000 abandoned github repos that implement one or two specific features in the commercial ones.

Devs(In general, not myself) don't want debuggers, they want printf. They might want Linux, but they probably like their own DIY hobby OS more.

thaumasiotes• 3 years ago

Employers do not value tools because the employees say the cost-benefit is not worth it. The problem is that the employees do the cost-benefit calculations remembering their time as a poor student when their time was worthless.
People in software do not internalize this because we can build our own tools.

I think there's another aspect. Those tools are doing things we can do anyway. Here's a story of something I did recently:

I wanted to download a series from viki.com. They have switched from just sending you the video file to sending you an encrypted video file along with the encryption key. This is "more secure".

So, there's a process involving detecting your own request for the encryption key so you can replay it and get the response, decrypting the video file, decrypting the audio file, and then muxing them together with the unencrypted subtitle file using ffmpeg. After figuring this all out (which took a significant chunk of time), I could manually run the process in 5 minutes for one video.

I wrote a powershell script to automate all the parts of it that weren't finding the URL of the three media streams, detecting the request for the encryption key or echoing the request and getting the key. Those stayed manual. But the script handled downloading, decryption, and assembly and it cut my time per video from 5 minutes to 2 minutes. That's a 60% decrease! But it's also easy to say "hey, 5 minutes was already pretty fast; what do I care about 3 minutes?". The payoff from automation only looks good if you're doing a lot of it.

(The automation also made it obviously worthwhile to go nuts on adding metadata to the final video file, where that would have been more annoying in the all-manual process.)

ericd• 3 years ago

My software hand trowels don't get randomly totally wrecked by some fresh grad UX guru who decided that they needed to rework everything to justify their employment or because they felt they urgently need to chase the latest design fads, because the old ones are so last year.

Most tools in those other professions don't have that problem. Once you buy them, they're generally static.

But yeah, your point about devs being hilariously stingy and not valuing their time is pretty spot on. I just don't think that's the main reason they don't want to buy commercial tooling.

xyzzy123• 3 years ago

The barriers are things like integration effort, carrying cost (keeping up with updates and the toil to keep the tool going), poor signal / noise ratio and subsequent triage and debugging effort, additional feedback to dev team (valuable!) but often with no additional time to address it (demoralising), driving workflow changes for whole dev team (not just 1 or 2 enthusiasts) is hard, training & learning costs, and finally an inertia effect - the business shipped successful product without $tool in the past (otherwise there would likely be no money to buy it) and this is a trivial proof that $tool is not "essential".

austin-cheney• 3 years ago

You must be in a line of work absolutely foreign to high level programming. In language ecosystems like Java, C#, JavaScript, and so forth people live and die by tools. You are not hired as an employee for your ability to program, but for your years of mastery on some bloated tool. Just look at any job board.

Jobs posted to HN by start ups are frequently comical in this regard. On one hand they claim to be solving tough problems and hiring only the sharpest people around, but more than half the time the hiring criteria just the same tool insanity as the corporate world. If I can make more doing half the work and dealing with the same stupidity in the corporate world then why would I want to work your start up? Maybe I just like being punished.

Talking bad about some tools often brings upon the same emotional fire as burning holy scripture in public. For many developers if not for certain tools, for example Java Spring or the big JavaScript frameworks, they could not program at all. They simply lack the skills to practice without such tools.

In these cases tools are more than valued. They are as essential to air and food. Its unfortunate, because economically that value is entirely artificial. An employer can easily eliminate these large tools, eliminate half the associated jobs, retrain the rest of the staff to move forward without those tools and pocket the difference without lose of productivity.

tj_astro• 3 years ago

Purify was a fantastic tool and I would have cheerfully paid $100K in late-90's dollars. We demoed it and the amount of memory errors it found in our legacy C and nascent C++ code stack was simultaneously depressing and exciting. We bought licenses and went to town. There was a markable product quality improvement as a result, at a time when our sales were exploding. I've never found a memory tool since that was as capable. We were running SGI IRIX boxes at the time.

version_five• 3 years ago

A venture capitalist once told me, “There is no money in tools.”

That suggests he's saying there is not upside in a portfolio of tool companies. Following average VC advice gives average VC results. If everyone listened to this kind of stuff we'd be awash is generic SaaS CRM stuff...oh

flappyeagle• 3 years ago

Tools matter where UX matters. Stuff that you set up once and a computer interfaces with in an automated fashion has no UX. That's almost the entire premium of every software tool in existence.

RagnarD• 3 years ago

Look at the catalog of ComponentSource (https://www.componentsource.com), which is mostly ... tools. Are Jetbrains and Progress (to name two offhand) doing badly?

thaumaturgy2• 3 years ago

I think about this often because it feels like my peak ability and productivity s a developer happened back on classic MacOS (7.5 through 9). There was a program called MacsBug that you could install, and with just two keys on the keyboard, invoke a stop-the-world debugger, complete with disassembly, stepping, and heap inspection. You could twiddle a bit here and there, even in the operating system. MacsBug shipped with a Forth interpreter, if I remember right, so you could even automate some of the debugging process. I still have my MacsBug reference guide.

I don't think there's a day that goes by that I don't wish I had those capabilities again.

But, operating systems also have another important function: they have to defend against malicious software. In the days of Macsbug, every program had full read/write access to every other program's heap. That would be horrifying today. Every tactic that's been developed in the name of secure computing has made OS-level debugging significantly harder.

dentalperson• 3 years ago

Thanks for reminding me about this. MacsBug was great. Besides being a useful development tool, MacsBug was used extensively to break shareware as well by the mac warez community. I remember seeing some guides about it that were very illuminating about how program execution and object code actually worked that made things click for me.

It was also a time when any consumer could issue a system interrupt with the same two keystrokes because it was built into mac OS, and see the cpu registers and enter hex to change the program flow. Since there was not as clean a system/program barrier as there was today and single threaded OS (I think?), before OS 8 the system froze all the time because of apps, and even non-expert users memorized the hex code for the system exit() trap.

thaumaturgy2• 3 years ago

0xA9F4, if memory serves.

guidoism• 3 years ago

The code used to build operating systems has only the most primitive of data types (lists, hashes, the occasional tree), while the libraries used in applications are a veritable cornucopia of modern data types.

Really? Is this only Linux (which gets a lot of eyeballs)? Mac OS? Maybe he's only talking about the kernel?

MobiusHorizons2• 3 years ago

He is definitely talking about the kernel (as well as things that run in kernel space like drivers). Userspace is all applications from this perspective, and can be debugged with a normal debugger.

n4te• 3 years ago

I like writing tools and make all my money doing that. It's a niche market, but it's mine and worked out amazingly well.

sky_rw• 3 years ago

It's worth noting that when a VC says "there's no money in X", their definition of what constitutes "money" is very different from a normal persons. It's more accurate to say "there are no wildly market-inefficient profit margins to capitalize on and make hundreds of millions from in tools".

Animats• 3 years ago

If you use an OS with a tiny, well-tested microkernel, you debug drivers and such with the user-level debug tools. I wrote FireWire camera drivers for QNX debugging them that way, and doing compiles on the same machine upon which I was testing. Without rebooting.

That's not new. The first user-space driver development was for the Michigan Terminal System on the System/360 model 67. That was the first computer with virtualization hardware. People have been doing this in IBM land for over half a century.

rhelz2• 3 years ago

The founders at a tool-making startup I worked for came back from a sales meeting one day looking very sad. I asked them why the long faces. They said they pitched the tool as something which would save their engineers a lot of pain.

"Eh?" said the prospect. "What? Who cares how much pain the engineers are in? Its their job to endure pain."

Clearly, the founders hadn't fully perfected their sales pitch....

AtlasBarfed• 3 years ago

This is very simple.

Tools in the "real world" generally work on one protocol: physics.

Yes.. there are standards and industries and the like in physical tools, but those are much more static and unchanging. Because physics and material world. A tool will generally be relevant until it wears out, and often/sometimes is useful for decades after "obsolescence".

In software, where (looks at the Javascript ecosystem) things change in a year. Sometimes less. Version churn and feature churn means your tool is obsolete/useless (if you are lucky) within 3-4 years. Often less.

In regular tools, it is pretty clear the distinction between a tool and the end product.

But in software, a tool is SOFTWARE, with all the problems of software: version churn, feature drift, underlying OS support, and all the other quicksand.

Tool markets are also inherently a fraction the endproduct market: The produced product volume is say 10x to 100x the value of the cost of the tool in the physical world before it needs replacement or wears out. Software being what it is, you can get 100000x - infinite the tool. That value proposition means the market for the tool is the inverse of the value factor of the tool. In software, you would get one tool and use it for 100000 products / views / etc. So the market for it is lower.

Perplexingly to my point about software tools going out of support, is that REALLY established tools are long lived and never wear out. So a hot gee-whiz tool will be competing against well-featured tools (often free after a while because tool vendors go out of business and open source it), so there isn't much gee-whiz anymore.

OpenAI is an interesting "tool". We'll see where it goes commercially. But most tools have reproducibility. AI tools do not, at least currently they don't.

Free software tools seem more durable, because the user population sustains it generally with contributions, bug reports, feedback, and demand. And sometimes the minimal money to support the org. And sometimes marketing.

Software is also generally about building a bandwagon. Gatekeeping tools are not good for bandwagon building. So software tools are generally free to make jumping on the bandwagon easier.

Stone knives and bear skins – there is no money in tools

Related Stories