Shipping Culture Is Hurting Us

http://bitbashing.io/2015/02/16/shipping-culture.html

by uam • 11 years ago

231 161 11 years ago

The author talks about languages and technologies (JS, Mongo), but he's really getting at something deeper. The real danger of the "ship it" culture is that things that can't be "shipped" right away — things that require solving really hard problems — tend to fall off our collective radar because there is just SO MUCH cool and (relatively) easy stuff to do right now. PG has a great term for this: "schlep blindness": http://paulgraham.com/schlep.html

digi_owl2• 11 years ago

Makes me wonder if this is why i find the whole "devops" concept to be raising my hackles.

saidajigumi4• 11 years ago

Maybe that depends on who's uttering the phrase and what it means to them. Some points in the space:

- "Devops" as a movement encompassing the ideas that developers should not be walled off from operational realities, and that software operations tasks should be encoded as repeatable, testable software (vs ad-hoc stuff a sysadmin does on a box somewhere).

- "Devops" as "oh look, we can hire less people and just get some 'devops' to do it all."

dkarapetyan3• 11 years ago

How about "full-stack developer"?

pjmlp4• 11 years ago

I agree. Almost 30 years into this and I still don't see the point.

It is almost as when cleaning became health assistance.

SeanDav• 11 years ago

DevOps is an integral part of Continuous Deployment. I am not sure why methods used to make deployment of software more frequent and less error prone should raise anyone's hackles. I guess it is not understanding what DevOps is and what it is trying to achieve.

hackuser• 11 years ago

Perhaps ship-it-now is the right answer for the times. With so many rapid changes in technology there should be 1) a lot of low-hanging fruit (i.e., quick, high-value solutions), making short-term project more valuable and 2) a shorter shelf-life for any solution (i.e., a new tech will make it obsolete), making long-term projects less valuable.

mattlutze2• 11 years ago

I wonder if ship-it-now isn't more a reflection of the times, rather than the right answer.

"Pick off the low-hanging fruit" works for a shop looking to turn some mad profit without having to do a lot of deep thinking. And that's certainly fine. But if that's the culture of the industry, we'd be sacrificing progress and innovation for a quick and ephemeral dollar.

There's a quiet theme that runs through the community here, and tech in general, that suggests everything that's new is often just old-again. A lot of these "rapid changes" really do feel like reinvention and change for its own sake, and seems plagued with the same issue as above -- building new iterations on existing ideas, picking off low-hanging fruit but not really going anywhere.

edit:

The problem is that these technologies, being so beginner-friendly and aggressively marketed, rapidly pick up steam and become the “cool” things to use, regardless of actual merit or lack thereof.

This line hit it on the head for me. I've been in the business for a while but have only been programming directly for a fraction of that -- I can readily admit, a lot of the newer JS libraries and frameworks made me feel like a superhero with almost no proper training or understanding of computer science.

adestefan• 11 years ago

The only reason why there is low-hanging fruit is because you need to reimplement solution in the tech du juor to problems that were solved 30 and 40 years ago.

laurentoget• 11 years ago

this sounds like a self-realizing prophecy to me.

technologies change quickly because everybody wants to ship fast and put out something just good enough to be better than the last half baked solution that was shipped because technologies change quickly.

how do you get out of that vicious cycle?

fijal• 11 years ago

The problem runs deeper than that even. Even if you like doing stuff that requires more effort, it's shockingly difficult to find places/people who want to invest/hire you to work on such technology. You have essentially a choice between academia (which is, well, sometimes very academic) and large corpos which have their own share of problems and not everyone fits.

hugs• 11 years ago

There is another option: work for a salary, live frugally, and stockpile savings. Then later (maybe 5-10 years later), work on the tech you really want to work on that has no obvious & immediate business need. The upside is that because no one invested or hired you to work on it, you own 100% of that work.

romaniv3• 11 years ago

And even in big corps, 95% of all IT is usually plumbing and hectically applying band-aids for past issues that should not have been there in the first place.

To me, the fact that IT is currently seen as rapidly progressing is laughable. It is rapidly changing, true, but for the most part we're solving old problems with new tools that work only marginally better than the old tools (because, like the article says, the underlying mentality is the same). Very often they change fast enough to keep everyone always "learning" at the expense of the users, so the marginal benefits are cancelled out.

(Seriously, I know people who made the same excuse for bad quality for the last decade. "Oh, I'm just learning [over-hyped tool]. Everyone makes mistakes at first." Learning tools is a fake kind of learning. Real advances happen when your conceptual basis is improved, so you get better at doing something with whatever tool. That's real learning. )

---

Also, sometimes we're using new tools that work significantly worse than some old tools.... because at some point conceptual development branched and "our" branch is at the same level of development as another branch was in, say, 60s or 70s. Sometimes I read about old tech and it just blows my mind. "What? They've been doing this three decades ago? How come this is marketed as awesome new thing right now?"

hiou• 11 years ago

I think many people confuse "ship it" culture with just doing crap work. I've seen too many ship it projects die because the biggest feedback from the customers was "This thing doesn't actually seem to work" or "This is useless". Ship it can apply to hard problems as much as easy ones, it's more about not doing extra work before you know what your customers want. But if you MVP it to the point where it barely works and has no real functionality then customers will often view it as trivial and useless and you'll likely confuse missing product market fit with under delivering.

mpweiher• 11 years ago

Yes. If your problems are essentially trivial and you are just trying to gauge customer moods, shipping and shipping again quickly is the way to go...and thinking about the problem will probably not help.

However, while quick iterations converge rapidly on a local maximum, they really, really suck at getting you out of that local maximum.

I notice this with trying to create a new and different programming language (http://objective.st): for almost all the problems I am tackling, there are quick and obvious answers...that get me stuck in the same mess we are already in. So I've found it necessary to deliberately delay implementing stuff, going slower than I can and make sure I leave the time for my not-quite-conscious thought processes to work out the problem and present the results during a relaxing shower.

danschuller• 11 years ago

An interesting read but poor tools can't only be attributed to shipping culture. A more important reason is the tools are good enough (only just, but that's enough) and at that point priorities change. And once momentum becomes big it's hard to change direction i.e. Javascript, so hard to revisit the fundamentals.

Better tools are coming, such as Lighttable / Bret Victor's talk: https://vimeo.com/71278954 . But it's not clear when they'll arrive! Will we still be developing web-apps in Javascript after another decade? It feels like we can do better.

ChuckMcM2• 11 years ago

The problem that I have with this thesis (that shipping culture is hurting us) is the results people have. You can certainly argue that a better result could be achieved by taking more care with your design, but it is unclear that in doing so you would have achieved something more "valuable" than you did by shipping and iterating.

I do agree with Gary Bernhardt that infrastructure is not getting the attention it deserves, but as we saw with GPG it isn't something that is easily funded. Sun paid an engineer who did mostly nothing but maintain xterm (and the Sun tools equivalent) for several years. Where are you going to find a sponsor for a new terminal? Current terminal code could be improved of course, but in so doing would you get better code? Better systems?

spitfire• 11 years ago

The thing is the "Think carefully and create deliberately" approach has been tried so very few times. But when we have tried it we've gotten Lisp machines, ACID RDBMs, strong typing, and many other important and very useful tools. I'd say it's worth a shot more often.

We know we're in a world of hurt now, but the argument against it is "oww. Devil we don't know!".

Math is hard, lets go shopping.

nostrademons2• 11 years ago

The "think carefully and create deliberately" approach has given us a lot more than that - there's also Smalltalk, Xanadu, Eros/E, NextStep, BeOS, microkernels, Plan9, Dylan, and General Magic.

A look at that list is pretty instructive for why more people don't take the "think carefully and create deliberately" approach: by and large, the creators of those projects failed to profit from their inventions. In many cases, they wasted years of their life slaving away in pursuit of perfection, and the market didn't care. If you study any of the systems I mentioned, you'll find some incredibly elegant and beautiful CS concepts, ones I wish I could use for every-day programming all the time. But the problem is that none of these innovations exist in a vacuum, and in the time it took to perfect the product, the market passed them by and the world changed in a way that made them no longer relevant.

ChuckMcM2• 11 years ago

"We" and by that I mean software engineers, used to do the 'Think carefully and create deliberately' all the time, we called it software architecture. But "They", and by they I mean software engineers with a 'ship it' culture, got stuff to market faster and improved it faster, and called it 'ship and iterate'.

Lots of people were called out as "old fogeys" or "dinosaurs" when they asked to think through some of the ramifications of shipping things. Those folks got trained that such behavior is a quick way to get managed out of an organization.

It would be interesting if, as an industry, wave 3 (which like wave 2 before it is a decimal order more engineers than its preceding wave) decided to go back to a more mindful way of developing software. And if so they could remain employed amongst the wave 4 cohort.

thaumaturgy• 11 years ago

There's also this really ugly aspect of the ship vs. art spectrum:

    ship ------------------------> art ------------>

At the most radical "ship it" end of the spectrum, you get things like nightlies. It's difficult to ship more often; you know when you're shipping a little bit too often because your bug database tells you. There's a real limit there, where you just can't ship it any more often than you already are.

At the "art" end of the spectrum, you can spend months refining a product and never really know if you're done yet. It's completely open-ended. Your bug database will never be empty, there will always be another feature request, and you can easily end up working on version 2.0 when 1.0 never even left testing. You can go forever in the "art" direction.

I agree wholeheartedly with the author's complaint. I spend most of my time interacting with end-users of software and other engineered systems. People are really frustrated. I keep trying to communicate their frustrations to programmers, but programmers keep blowing it off: "oh, people just hate change, they'll get used to it"; "they should update more often, we fixed that bug right after the software was released".

I used to expect there'd be some kind of backlash at some point, but now I think it's worse: a lot of people gave up, they just don't expect trouble-free software anymore. It happened just the other day with a bookkeeper who visited a client's office while I was there: "this will just take a minute ... oh, Quickbooks updated ... oh, huh, it needs me to re-enter all that information I entered a while back ... I'll have to look that up ... oh well."

And it's not just the end-users. Programmers expect software to be broken too. They almost relish it, it seems. "That's just how it is, fix the bug yourself" or "it works for me" are both common responses that completely dismiss complaints from other programmers.

New terminal code could be really awesome. I would love to be able to just drag-and-drop files between terminal windows and have them automagically scp stuff between servers. I'd love to have the ability to open a remote server log file in my local text editor and get everything syntax-highlighted for me -- tailing mail.log with live syntax highlighting in Sublime? Oh yes please.

I do think some infrastructure is getting attention. Containers are (maybe) some progress, AWS has been a huge revolution for a lot of people. But yeah, there also seems to be a lot of popular technology right now that isn't really advancing the state of the art very much, while a lot of nuts-and-bolts parts of the industry are really suffering.

...I forgot what my point was supposed to be.

wodenokoto• 11 years ago

While it is great using hyperlink to point readers to sources, it shouldn't become a substitution for actual words.

DiThi• 11 years ago

Instead we get some bizarro-world where where the type of NaN (“Not a Number”) is number, where NaN !== NaN*, and a chart like this exists for something as simple as comparing two values.

That's the definition of NaN in virtually any programming language with floating point numbers. And the comparison table makes sense with the rule "when types are incompatible, both are casted to strings". Just use === instead of ==, which is only really useful for comparing against null/undefined.

andrewstuart24• 11 years ago

Also in logic. Not something is never equal to not the same thing.

A bird is not a rock. A computer is also not a rock. Clearly not equal to each other.

mxfh• 11 years ago

I think people confuse the nature of NaN with the result of isNaN() a lot, this being the source of this confusion.

kazinator14• 11 years ago

The stupid thing about IEEE NaN is that it's not equal to itself! If variable x holds a NaN, then (x == x) is false.

This violates: http://en.wikipedia.org/wiki/Law_of_identity

If (x == x) tests false, then it asserts that x is not itself, which is logically preposterous.

ANSI Common Lisp has a bit of this problem in it too, but it's not required; it is there for some weird historic implementations. That is to say, if x holds a number like 1, then (eq x x) is not required. (But in sane implementations it does yield t; and it yields t even if x is a bignum, because (eq x x) is given the same object as two arguments. Two separately computed bignums of equal value will likely, of course, not be eq.

How this can be explained is that eq tests "implementation identity", and somehow different instances of a number are treated as different implementations. Argument passing is by value, and the two reductions of the expression x in (eq x x) to a value somehow produce a different implementation of the value.

This rationale is unrelated to IEEE NaN-s, though.

pwnna• 11 years ago

Don't quote me on this, but I recall the rational for NaN is because NaN is typically the result of a division by 0.

Divisions by 0 can be thought of as infinity (for the sake of this explanation, but mathematicians will cringe), but it is not any particular infinity. In the sense that x / 0 does not necessarily have to equal y / 0. For that definition, the result of a division by 0, NaN, must not equal itself.

You can use `isNaN`, tho.

cwyers2• 11 years ago

NaN literally means "not a number." Lots of things aren't numbers. The letter a is not a number. The square root of negative 1 is not a number (at least, not one representable in floating point math). a is not equal to the square root of negative 1.

mherkender• 11 years ago

Having consistent behavior for NaN means that I can write "<math operation> ? x : y" and know that "y" will be the result when "<math operation>" involves NaN. It's useful.

Not saying it's the perfect way to handle things, but it's pretty reasonable in world where people write code that generates NaN almost every time they use division. I worry more about writing code that generates NaNs than NaNs themselves anyway.

aidenn02• 11 years ago

You are missing a lot of nuance here.

First of all it is debatable over whether or not the spec allows (eq x x) to be false. eq is required to return true if the arguments are the same identical object. It would be a twisted interpretation of that to allow (eq x x) to ever return false.

Now (eq 1 1) is specifically not required to return true, as, for example an implementation that boxes all numbers could create two separate objects for that expression, and the arguments are now in fact not the same identical object.

This is something that exposes implementation details to the user, so using eq is discouraged (and in fact it can only portably be used to compare symbols). There are times when the best way to accomplish what you are doing is to (ab)use implementation specific behavior, and this is particularly true of Lisp which was a dynamically typed garbage collected language in the 70s (yes, predating the VT100 terminal referenced in this article).

nemothekid• 11 years ago

The stupid thing about IEEE NaN is that it's not equal to itself!

Why would it be? The rationale makes perfect sense. It would be unexpected to believe that 17/0 == 0^0. Both are NaN, and equally nonsense, but 2 very different statements.

Secondly, IIRC, NaN is supposed to break assert(x==x), because if you use NaN in your program, then your program is undefined.

jks• 11 years ago

According to http://stackoverflow.com/a/1573715 the IEEE-754 committee decided to make NaN != NaN in order for programmers to have a simple way of detecting NaN before there was a standardized isnan function or macro.

laxatives• 11 years ago

The law of identity shouldn't apply to NaN because it isn't a single value. NaN represents a set of values that get mapped to NaN. But information is lost along the way. Since we can't recover that information, we can't know for certain what value this NaN represents or whether this NaN represents the same thing as that NaN.

mbrubeck• 11 years ago

Yes. The research culture that produced Haskell, for example, is far from the "shipping culture" that produced JavaScript 1.0, and yet:

    > let x = 0/0
    > x
    NaN
    > x == x
    False
    > :type x
    x :: Double

(and yes, Double is an instance of the Num type class)

The author programs in C++ and D. I wonder if he knows the value of (NaN == NaN) in those languages?

thinkling• 11 years ago

I think the author just picked an awkward example. The real point is the existence of that table of weird (non)equalities and (non)truthinesses in JS [1].

[1] https://dorey.github.io/JavaScript-Equality-Table/

romaniv3• 11 years ago

NaN behavior might actually be appropriate in Haskell. I don't know enough about it to comment. But it is not helpful in JavaScript. Part of the problem is that in JS you can get NaN in a variety of different ways that do not involve mathematics. And that all numbers can be floats (whereas in other languages we have integers). And that NaN can be silently propagated and morphed in in a more complex expressions.

angersock3• 11 years ago

So, there's a great argument to be made about the negative impacts of "Shipping Culture".

One could argue that it encourages underdesigning of ultimately complex systems. One could argue that it encourages excessive code and infratstructure reuse, to the point that even trivial projects pull in more than is needed and consume more resources than even remotely necessary. One could even argue that it creates in customers an unreasonable expectation about ongoing fixes to a project, about ongoing support contracts and culture, and generally discourages creating artifacts that later generations can use in favor of ongoing maintenance performed by morlocks.

Sadly, the author makes none of those arguments, and instead gripes about hackathons, about Javascript (badly), and about MongoDB (which is fun, but doesn't go anywhere).

Author bemoans "ship it now" culture, bemoans shipping things before they're technically good--and then has the gal to use a Netscape Navigator screenshot! Does anyone else find irony in that? Netscape was never known for good quality code or sane implementations--go read old kvetching by Brendan Eich or Jamie Zawinski. You know what they did do, though? Fucking shipped.

Author complains about NaN--which is a goddamn standard in IEEE 754, which is exactly how C++ (their pet language) handles things, and which is completely reasonable behavior for any JS implementation. Not having an integer type? Newsflash kid: JS supports bit-accurate integers up to Int32s.

Author claims systems stopped using coopterative multitasking two decades ago. Author has presumably never written code for hard-real-time embedded systems.

Author goes on to complain about ancient standards. I have an ancient 120v 60-hertz AC circuit running my house. I have an ancient system of measurement for the beer on my desk. Funny thing about ancient systems: if they're still around, it's because they've perhaps solved the problem well enough to stay around.

"Shipping Culture", as defined by the author, isn't what's hurting us: it's the influx of jackasses with loud mouths and no appreciation for history, business, or engineering loudly proclaiming that perhaps the most productive decade in software engineering is somehow wrong.

91bananas• 11 years ago

All I can say is, thank you.

username2232• 11 years ago

Yeah, the title and the article have very little to do with each other. Complaining about NaN semantics is especially irrelevant.

OTOH, it's funny you bring up Netscape. Yes, they shipped. Back in the Netscape 3-4 days of the Browser Wars, every new version was a crap shoot. It might un-break some page you wanted to use, but it might also be drastically slower, crash a page that you needed, lose your preferences, or whatever.

But the cool thing is that you could choose when or if to update! You could install the new version alongside the old one, try it out for awhile, and junk it if it was worse overall. If most people thought Netscape was getting worse, then most people would stick to the old, working version. Nowadays browsers follow the same "random walk" development model, but shove the latest version down most users' throats whenever they feel like it, and make it hard or impossible to revert the latest breakage. "Ship crap" combined with "software force-feeding" is good for no one but lazy coders.

angersock3• 11 years ago

So, on the one hand, I agree very much that having the options available was pretty cool.

On the other hand, having seen this repeated time and time again in the enterprise world, I don't think users should be given the option of not updating, especially for services that they aren't hosting themselves. All they end up doing is creating a support burden and being dissatisfied.

Our users are increasingly ignorant about the systems that they use--I think that precludes them from having final input about how said systems are implemented and deployed.

andrewstuart24• 11 years ago

Every time I see someone quote that JavaScript was designed in ten days I cringe.

JavaScript was designed in 1995. It was standardized as ECMAScript in 1997. Ever since then, it's been under active development by a thriving community of engineers pushing for better standards.

It's been a bit longer than ten days.

tedunangst• 11 years ago

How many of the first ten days design decisions have been reversed by subsequent standards?

austinz• 11 years ago

Given that the ECMAScript community has (for good reason; "don't break the web") decided to avoid backwards-incompatible changes, that necessarily places a limit on the amount of "fixing" they can do to the language. New or enhanced features are okay, rectifying mistakes is harder.

(This isn't just a JS thing; go talk to any random Python developer and ask them what they think of Python 2 vs 3. There are tradeoffs to both approaches.)

http://www.2ality.com/2014/12/one-javascript.html

CmonDev4• 11 years ago

"don't break the web"

I know it's not your fault but this statement doesn't make any sense. Web is made of hyper-linked content not jQuery animations and AJAX.

moe• 11 years ago

Ever since then, it's been under active development

What have been some significant changes to JS since it was thrown together by Netscape?

It seems to be largely the same trainwreck today that it was 15 years ago.

andrewstuart24• 11 years ago

http://en.wikipedia.org/wiki/ECMAScript#Versions

http://kangax.github.io/compat-table/es5/

shawndumas• 11 years ago

what would be helpful is a link to your language for comparison.

bsder2• 11 years ago

Every time I see someone quote that JavaScript was designed in ten days I cringe.

I cringe because it's not true; even the original Javascript wasn't really designed in ten days.

If you look at the history, Eich was playing around with designing a language for months before the order to shove Javascript into Netscape Navigator. Netscape had large software libraries for dealing with virtual machine-type problems for dealing with Java, and Eich was quite familiar with them by the time Javascript was "designed".

BrendanEich2• 11 years ago

Quit your fibbing! I joined Netscape on April 4, 1995, to "do Scheme in the browser". Immediately I was out of luck on several fronts:

* Put on the server team with the McCool twins (NCSA httpd and then Netscape's reboot of same) and Ari Luotonen (proxy guy), working on "HTTP1.1-lol".

* Told Java was in play with Sun -- the deal was not done but it was likely to go forward in Netscape 2 -- so maybe never mind about Scheme, but:

* If there was to be a "scripting language", it had to look like Java.

Kipp Hickman (Netscape first floor, and my kernel hacking colleague from SGI) and I wrote the "Netscape Portable Runtime", NSPR 1.0, the "large software libraries" you allude to, in April and May.

Kipp used NSPR for his Java VM prototype, which Sun helped convince him to abandon (single-source implementation required or else bug-for-bug compatibility would be a nightmare).

So, much of that "dealing with virtual machine-type problems" code was from me, not from some anonymous and non-existent team at Netscape who had "months" ahead of May to develop at leisure.

When I switched to the client team in early May, @pmarca and I had been conspiring (with Bill Joy of Sun on-side; he signed the trademark license for "JavaScript" in early December, 1995). Marc made the case for "a language you put directly in the HTML" -- not something you compile into an applet. This idea got enough support for me to spend ten days, a week bracketed by weekends without much sleep, hacking the first "Mocha" runtime.

I spent the rest of the spring and summer embedding that Mocha, then LiveScript, interpreter into Netscape 2's rendering engine and network library; I had help from the front end hackers (@jwz and Spence Murray on the XFE; @knobchouck and Garrett Blythe on WinFE; Aleks Totic on MacFE), who did all the native control integration; we collaborated on the front-to-back-end API.

During these months, the only file I didn't write in the original JS implementation was mo_date.c, the Date object. Ken Smith, who joined from Borland with three others as a team, helped do that by closely porting Gosling's JDK1.0-prerelease java.util.Date code from Java to C.

Why you are making up facts now, I have no idea. If you have some overriding animus against JS or me (or both), take up a better argument than making up false history. That inevitably blows back.

If you can't believe I wrote as much code as I did, you should see what TJ Hollowaychuk has done in the last few years. But yeah, I was writing lots of C code then.

If you want to see others testify to my account-in-brief given above, check Quora (http://www.quora.com/In-which-10-days-of-May-did-Brendan-Eic..., http://www.quora.com/Did-JavaScript-evolve-from-another-prod... [other witnesses testify there], http://www.quora.com/How-did-ScriptEase-influence-the-design... [refutes an obsolete fraud to-do with NOMBAS]), listen to the JS Jabber show from last fall, and ask anyone who was around then. I've given some twitter handles above.

_pmf_3• 11 years ago

Every time I see someone quote that JavaScript was designed in ten days I cringe.

Well, the fact is, this does not seem unlikely.

kazinator14• 11 years ago

This is not just the last several years.

It comes from a slogan that was popularized by Eric S. Raymond some 18 years ago:

http://en.wikipedia.org/wiki/Release_early,_release_often

irickt• 11 years ago

Or Guy Kawasaki, circa 2000

>> Don't worry, be crappy. Revolutionary means you ship and then test... Lots of things made the first Mac in 1984 a piece of crap - but it was a revolutionary piece of crap.

d_b42• 11 years ago

All things in moderation..

To me, "ship it" as a philosophy means the most important thing we do as developers is deliver working code to customers.

On my team, when we encounter a concept that is just too complicated to ship right away, we try to branch-by-abstraction and keep shipping changes, even if the feature isn't "ready" yet.

To me, the marginal returns on trying to be right diminish more rapidly than the returns on making it easy to fix to stuff when I'm wrong. Maybe I'm just not as smart as everyone else here, but it seems to work.

Sir_Substance• 11 years ago

It's more of a business necessity then a philosophical point of view.

The history of software development is littered with companies that failed because they spent so long polishing the beautiful crystal tower they were developing that someone released a concrete one instead and everyone bought that, and then no one needed a crystal one any more.

Unfortunately, it can be really hard to find out about these companies because of how hard they've been trodden into the ground by their successors, but Xerox and Netscape would be two high profile examples.

Shipping constantly may not be the best thing for technology, it may not be the best thing for quality and it may not, at times, even be the best thing for your users. But if you don't, someone else will.

saidajigumi4• 11 years ago

This whole article makes me want to scream, if only because the author seems to have never heard of the concept of Path Dependence[1]. I could utter a similar rant, on how terrible it is that we're stuck with awful legacy dumb AC lightbulb sockets everywhere, and wouldn't it be nice if the entropy fairy just waved a magic wand and we all had 'net-connected DC smart sockets for our wonderful Future Bulbs.

But that kind of talk utterly dismisses the important reality that 1) we don't have a magic wand and 2) we have to deal with things like path dependence and network effects as phenomena. Example: if you chart out historical network bandwidth, you'll get a Moore's-law-esque curve, but with a significant step function depending on which network aspect you study. Why? Path dependence and (literal!) network effects: we don't see the benefits of network bandwidth improvements until enough hardware has been upgraded to see an end-to-end improvement.

[1] https://en.wikipedia.org/wiki/Path_dependence

soup102• 11 years ago

Javascript has become the defacto browser programming language, even though it's objectively awful for complex software. The only reason it's achieved it's high status is because browser developers refused to cooperate and develop something better. Microsoft and Apple view software lockin as a competitive advantage and have actively undermined technologies they thought were threatening(java and flash amongst others). The explosion of Html5 and javascript was more of a 'nature will find a way' type development and can almost entirely be credited to Google and various web companies pushing it and making it better.

It's still a very flawed technology stack for complex web-apps though and it's very reasonable to suggest that we could do a lot better.

nulltype• 11 years ago

If you think Javascript is bad for complex software, you should try C++.

nostrademons2• 11 years ago

Grandparent's point is that any model of reality that doesn't acknowledge the fact that sometimes powerful actors will refuse to cooperate is a flawed model of reality.

Who's the "we" in your comment and the original article? HN readers? The programming community at large? You and anyone else reading this article are free to write a compiler for a much better language than Javascript. There is no guarantee that Microsoft or Apple or Google will not actively undermine you too if it looks like your application is becoming a threat. More likely, such a language wouldn't serve the needs of most developers, and it'd be ignored like many other brilliant research languages that cannot get popular adoption.

saidajigumi4• 11 years ago

it's very reasonable to suggest that we could do a lot better.

Yes, and it's so damn obvious now that you'd better have something salient to add. To wit: this has been done much better by others. C'mon, the book is called Javascript: The Good Parts (cough, 2008 vintage) This post is just chock full of bizarre straw-men in the guise of actual argument:

Systems stopped using cooperative multitasking at least 20 years ago because it sucked compared to the alternative of automatic, preemptive multitasking. And yet Node.js harks back to those dark days with its callback-based concurrency, all running in a single thread.

What the heck? No comprehension of evented systems? Reactive programming, anyone? No acknowledgement of the use cases when these systems do (or do not), in fact, kick total ass? Yes, the vanilla JS nested callback thing is a bit annoying. So don't do it. Use some of those FP chops and unwind it. Maybe a fold or a monad or some categorical pocket lint. Just please let cooperative multitasking have a nice rest.

dasil0032• 11 years ago

The explosion of Html5 and javascript was more of a 'nature will find a way' type development and can almost entirely be credited to Google and various web companies pushing it and making it better.

You say this with a pejorative tone, but the fact is I've yet to see a human-designed system that is as elegant or as robust as the systems evolved by nature.

Sure for relatively simple problems in isolation we can design extremely elegant solutions, but complex systems beyond the scale of one human mind become exponentially more difficult to properly design as communication overhead quickly overwhelms the capacity to reason about the entire system. Abstraction is our main tool to fight this, and it is powerful, but most abstractions are leaky, and there's always some essential and irreducible complexity that fights against encapsulation and modularity.

As long as I've been a web developer (20 years) there has been this constant drum beat of hand-wringing over the terribleness of the web and its "abuse" as an application platform, all the while its evolutionary traits as a simple, open platform which has achieved an ubiquity that no single vendor platform has ever approached. Think about it: web pages are accessible to people with any disability on almost any device in the world regardless of operating system. Yes, the win32 API is a more elegant substrate to program a GUI, but even with in the flower Microsoft's dominance it still paled in comparison to the ubiquity of the modern day web. Fixing the warts of the web is not the hard part, the hard part in designing a "better" platform is achieving adoption, that would require an act of god as there is no coalition powerful enough to make it happen through human volition alone.

andrewstuart24• 11 years ago

Funny you mention flash, since any logic in flash is done in ActionScript, an ECMAScript variant.

Perceval• 11 years ago

…browser developers refused to cooperate and develop something better. Microsoft and Apple…

Javascript was the language for nearly a decade before Apple shipped Safari.

Netscape shipped Javascript, but no one was necessarily locked in. When Microsoft shipped IE, they included an only somewhat compatible version of "JScript"/"DHTML" in it, with proprietary extensions added in. There wasn't even cooperation of Javascript per se, much less a "better" language to replace it.

bo1024• 11 years ago

I don't understand your complaint against the article. I think it exactly discusses the harms arising from path-dependence and how we need a lot of focus, dedication, and hard work to overcome this path-dependent inertia, to build better tools, and to get them adopted.

romaniv3• 11 years ago

we don't have a magic wand and

Many of the problems he describes do not need a magic wand. In fact, many cases of horrible tech are were created by making an arbitrary choice between several possible options which contained far superior solutions. That's what why the situation is infuriating. Fixation on bad technology is often presented as equivalent to consumer-grade standards (wall plug, lightbulb socket) when it is absolutely, totally different in nature.

cageface• 11 years ago

You see the same thing in natural organisms. Vestigial adaptations and odd or even awkward designs abound, yet they are functional. Some people take "Worse is Better" to the extreme of "Worst is Best" but there is some wisdom in the idea that it's better to get an adequate solution into play early than to wait for for a perfect, clean-room design. The important thing is to setup a feedback loop that continually refines your solution.

For an inspiring take on the power of chaos in evolution check out Errol' Morris' film "Fast, Cheap & Out of Control": http://en.wikipedia.org/wiki/Fast,_Cheap_%26_Out_of_Control

kazinator14• 11 years ago

Systems stopped using cooperative multitasking at least 20 years ago because it sucked compared to the alternative of automatic, preemptive multitasking. And yet Node.js harks back to those dark days with its callback-based concurrency, all running in a single thread.

Uh, no. For instance, Unix kernels have been traditionally cooperative---at least when executing kernel code. That is to say, user space can be preempted but not the execution of kernel code. The introduction of SMP brings concurrency into the kernel, though, and with that, preemption can follow. Linux development followed this path.

Still today, you don't have to turn on CONFIG_PREEMPT when building your kernel. If you don't have SMP either, then you have a cooperative kernel: one task is in there at a time, and it has the CPU until it voluntarily calls into the scheduler.

Cooperative tasking has the enormous advantage that it makes a whackload of potential race conditions go away, and that could be a possible reason why Node.js is the way it is.

Unix might not have been successful had Ken Thompson decided to make the kernel preemptive, and then spent 1973-1987 debugging it. :)

pjmlp4• 11 years ago

Unix might not have been successful had Ken Thompson decided to make the kernel preemptive, and then spent 1973-1987 debugging it. :)

Sadly that didn't happen, UNIX was adopted by 80's hipsters into their startups, became adopted by the industry at large, thus spreading C into the industry.

Now we have patch <choose day of week>.

rjblackman• 11 years ago

people working on computers in the 80s were not hipsters.

cben• 11 years ago

I'll bite. When I learnt C I found it charmingly simple and it took me years to realize how deeply broken some aspects are, especially the separate compilation circus — programmers working hard to help the compiler and not getting any modularity benefits in return.

So suppose C didn't happen. Do you expect the industry at large would have adopted a great language instead? Or some random product of history, half way between PL/I++ to JavaScript?

McUsr4• 11 years ago

I haven't tried Mongo.db, but Node.js takes up a lot of resources in a browser, Node.js apps feels a bit like the widest book strategy in the bookstore, you have to shove other stuff out of your browser, in order to use stuff that uses Node.js, and I feel that as a rather arrogant approach to deliver software, especially, when the functionality of Node.js apps doesn't really dictate the necessity.

AS for Javscript, there are worser languages out there, and the debuggers are good enough now, so it can be perceived as a "normal" language.

computmaxer• 11 years ago

but Node.js takes up a lot of resources in a browser

You must be confused. Node.js is used to run JavaScript on the server, not in a browser (the client).

McUsr4• 11 years ago

That may be so, but when I run something with Node.js in my browser, then I may have to close other things. So, wherever it runs, it really uses too much resources, and therefore I find it as a bad idea from the consumer perspective. It shouldn't be so, that somebody else, should dictate the content of your browser.

rudyjahchan• 11 years ago

You should switch to io.js. It's more performant. :trollface:

narrator• 11 years ago

People use mongo and node because they want to get a site up really really fast and see if it gets traction. When they actually get somewhere they'll rewrite it in a better language.

If you want to set yourself up with really slick tooling and a great language you can code in Scala with Intellij. You can even avoid touching any ugly dynamically typed stuff by coding your JS in scala.js. If you're doing it better than the other guys then great, you can code rings around them. Maybe it's not worth all the extra complexity when just starting out though.

When you get to big corp size code bases static languages are more common. That's because to be able to navigate and maintain that big of a code base, a good IDE, a static typing compiler and refactoring tools are a huge help.

edejong• 11 years ago

The interesting thing is that all these big-corp code-bases started out relatively small. The origins were actually able to overcome the complexity-wall that arises when a product is successful.

falcolas• 11 years ago

When they actually get somewhere they'll rewrite it in a better language.

Anecdata for you: Not necessairly. I'm working for a company which has "actually gotten somewhere", and our development team is still writing in Node with Mongo.

Of course, there is a reason for it, the front end development team has spare cycles, and the backend team does not. Ergo, Node!

alxndr• 11 years ago

I'm inclined to repurpose a cliche, and say that now your backend team has two problems!

chvid• 11 years ago

I wonder if "hurtful shipping culture" has a parallel in the blogging world?

mrgriscom• 11 years ago

Reminded me of this essay: http://www.jwz.org/blog/2012/08/a-generation-lost-in-the-baz...

I don't think it's fair to attribute the issues he brings up to "shipping culture". Rather, the causes are two-fold:

1) Lack of appreciation for history, being doomed to repeat it, all that

2) Making anything above the level of 'just not-terrible enough' without a benevolent dictator is hard.

igl2• 11 years ago

The problem is that the Author is assuming that 90% of software we write to make a living is set into stone and suppose to run forever. Almost everything we do is disposable and replaced within the same decade.

So he does not like Javascript and Nosql. Is it hard to find a JavaEE job? I think not.

hyc_symas2• 11 years ago

I think the real problem is the disposability attitude you just described. It's prevalent in our real world consumerist economy and it's killing us there too.

Building software that's to be thrown away is a waste of mental resources and physical resources. Society and civilization advances on top of our lasting creations, not the ephemeral ones. Reinventing the wheel doesn't advance the state of the art. You want to build one good set of tools that will last a long time, so you can stop thinking about them and be free to tackle the next truly new challenge. Doing anything else is just a waste of life.

igl2• 11 years ago

wat? How can you even compare the "problems" of building commercial software to the problems of consumerism in capitalism?

The standards applied here are whack in the scope of what js and mongo are used for.

Also: You only get better by reiterating.

yason• 11 years ago

Shipping doesn't always need to mean shipping to the public.

I write complex software too but I "ship" to myself early and often. I start by trying to solve a hard problem as fast and as quickly as possible in a naive way so that I can experiment more freely. This gives me more insight into the problem itself and into the solution space. The solution often changes -- sometimes it's the problem that changes.

If I were working with a customer or a startup I would definitely "ship" early and often, too, but I wouldn't give much weight to these "releases". They're just something to play with and to cut the shape for the problem.

halayli• 11 years ago

It's a spectrum. Trying to ship a perfect software from the start will probably end up like OS/360 project. Then K&R came along and designed what we know now as unix. An OS that wasn't complete and fully featured like OS/360 was intended to be, but it worked!

Making the wrong choices is inevitable, but correcting them is part of the software life cycle.

We aren't working with concrete and building bridges here. If we built Pisa tower like software, we can still fix it.

So yes, ship when you can as long as it works and delivers what it promises. It doesn't have to be perfect.

hyc_symas2• 11 years ago

Delays getting OS/360 were a blessing, really - it prompted many universities to develop their own mainframe OSs, thus helping OS research flourish around the world.

erikb• 11 years ago

I actually watched the screen cast this blog post is about. [1] And I would like to say something about that here, because the questions from the screen cast were also coined in the blog post.

The Question is this: "Why do people not replace VT100 style terminals?" There are two reasons. tl;dr Terminals have other reasons than programming in them and people actually reinvent how you can program on a daily basis. See more in depth arguments below.

The first thing is that people actually still need the old terminal stuff. There are loads of old computers you want to communicate with (just think of your pa trying to relive old times be trying to get the 80's game console to work). And there are also a lot of technologies that really need something that stupid, e.g., if you develop your own embedded system you might actually communicate with it using these VT100 commands. So yeah. Wow. A Terminal (emulator) doesn't have the task to show your text editor. You can start GVim if you just want to show your editor. They have the task to communicate. They can communicate with the system you are running them on or you can use them to communicate with another terminal. If you want to replace them, you have to replace the software in your pa's console (and probably some hardware), you have to find a new way to develop fresh, small computers, you have to find a new way SSH works, you have to find a new way to show your text editor. It's not impossible, but it's probably so hard that nobody would like to spend their whole life (work time, spare time, youth to death) doing it. It's not worth that much. Summary: It's not worth it, if you consider the whole picture.

And second argument: People actually do, if you just think about use cases like coding, gaming, etc. A modern game doesn't run in a Text shell as e.g. Nethack could. It runs in a graphical shell and is represented to you in 3D, e.g., GTA 5. Also there are many people who use IDEs. Unix+Bash+Texteditor was actually an IDE. Eclipse etc are a new way of thinking about the editing task with helpers like compilers, static analysers, debuggers, performance analysers, unit test runners, etc. There are even people who reinvent the programming wheel from another point of view, e.g., have a look at NoFlow. The reason the other stuff is not dying is because it's useful for other reasons. That doesn't mean you have to still use it for programming (also some people, like me, choose to). Summary: People do work on finding modern ways to program.

[1] https://www.destroyallsoftware.com/talks/a-whole-new-world

annnnd• 11 years ago

While I agree with the facts in the article, my own conclusions are quite the opposite (feel free to downvote / flame, but please read the whole response first).

I guess it depends on values - for me, what matters is the value added to customer. Iterating quickly (often) yields better results in this area because it allows you to guess customer expectations early in development process. I have often encountered die-hard engineers who want 100% specification upfront... In my experience, world doesn't work this way. It would be nice if it did, but it doesn't.

And yes, I am guilty. I am using MongoDB AND JavaScript (but not Node.js - not that I have anything against it, just never had the need for it). I don't use these technologies because they are "cool", but because they solve specific problems in an efficient way. Which is probably why hackathon devs used them too. And yes, I would appreciate schema in MongoDB and types in JS, but I can live without it. Life is made of tradeoffs. Does it really matter if latest Fart App (tm) builds on transaction-safe DB and uses strong type language?

So, are we seeing a rise of "shipping culture"? Yes. Does it change how we work? Yes. Is it in some ways worse? Yes. Is it hurting us? No.

foombarder• 11 years ago

Oh yeah, right - I forgot that we write code for the sake of engineering - why would you wanna get stuck in the shipping loop? ...

logicallee• 11 years ago

honest question. do you guys want a piece of crap from me that illustrates a new application type? (the way bittorrent or bitcoin or napster or wikipedia was new?) or should I wait.

I don't have any resources to put into this but can release a piece of crap myself. (I don't personally program professionally.) honest question - discuss.

would you like a piece of crap - or for me to wait. (Nobody more competent is going to just code this for me, at least not until the piece of crap exists and has traction.) I don't really envision other options but am open to them. What should I do? Get it right (not happening) or get it out?

jtth• 11 years ago

im from tumblr and what is this

whiskeySix• 11 years ago

The first two comments in the linked article are gold.

EGreg• 11 years ago

I would say Ship Often and A/B test. And before that, get really big test pool!

The first part to success is making it safe to fail

dustingetz• 11 years ago

Seth Godin wrote that 5 years ago, i think (hope) the economics are shifting, in the last 5 years a lot of the omgcats apps (market opportunities that can be captured with lousy software) have already been written, i think it may not be so easy for companies started today. However with the pace of innovation in the last 5 years (basically the coming of age of functional programming) it's a lot easier for a single person to write high quality software.

Anyway common wisdom is always going to lag behind today's actual best practices

RangerScience• 11 years ago

The author's bit about how we forget about the problems and solutions of our forefathers - that resonates with me.

We do seem to keep re-inventing the wheel, and while part of that is out of joy of creation, I think a lot of that is that we're not writing down the problems, and we're not teaching the problems. We're only paying attention to solutions, and that's limiting.

Code is a solution to a problem, but it's not always apparent what that problem /was/ when you just look at it. So NodeJS (according to the authoer) is doing something that was thought of as a good idea at first, and then people learned why it wasn't, and /nobody wrote that down/. Or, at least, when they wrote it down, nobody taught it.

Solutions are /answers/, problems are /questions/. Here's a beautiful illustration of this dichotomy: http://dead-logic.blogspot.com/2012/09/a-collection-of-quest...

This is then part of why the TDD movement is in the right direction - you write down the problem you're going to solve, and later, someone can come read it. And maybe teach it.

(Not saying TDD has gotten "there", but it's in the right direction)

Shipping Culture Is Hurting Us

Related Stories