Seeing Like a Programmer: Resiliency, Limits, and Moral Hazards

https://v5.chriskrycho.com/elsewhere/seeing-like-a-programmer/

by todsacerdoti • 2 years ago

76 43 2 years ago

v5.chriskrycho.com

Sorry but the arguments here seem poor quality and unconvincing. I suggest never beginning an introduction with "this is the best talk I've ever given".

The points about functional programming are bog standard, the characterize of "modernism" is extremely thin ("trying to master the world"?) and the connections between the two topic thinner still, if that's imaginable.

chriskrycho3• 2 years ago

I suggest never beginning an introduction with "this is the best talk I've ever given".

Heh, this is fair, and I actually just edited the intro in response to that feedback. In my defense, the talk itself doesn’t start that way; that was how I introduced for the folks who read my blog via feed or email subscription. Might need to iterate on having ways to add “feed-only content” for that kind of thing, because I think it’s reasonable to say that in the context of folks who already follow your work, but you’re right that it’s weird at the start of a “regular” blog post for people coming to it fresh!

Sorry to hear the rest of it didn’t land for you. Can’t win them all!

joe_the_user2• 2 years ago

I hate being "mean" but I'll admit sometimes my frustrations get the better of me.

I am interested in efforts to connect the act of programming with the large issues of human existence. However, I tend to also be critical these approaches also have the danger of creating shallow, surface comparisons that don't actually provide much value.

In this case, I feel "high modernism" as a concept loses most of it's content if one removes all particular historical strands that made it. Applied in the abstract, it becomes "central control bad, M'kay".

The interesting thing is you describe a lot of programming programs reasonably but it seems obvious to me that the answer to them is ... centralization and control. The problem of many systems is that software gives the impression that everyone's needs can be accommodated simultaneously and so results in actually contradictory requirements - especially when there isn't central control of the requirements. I'm sure that could be spun as too much central control too but that spin just seems unnatural.

commandlinefan• 2 years ago

For the most part it's good, but impractical, advice, unless you have "my dad owns the company" type job security. "Let it crash" is correct, but your (non technical) boss is counting how many times it crashes and putting the people with the most crashes up for the next round of "regrettable" layoffs. Even people with advanced degrees and decades of experience are just barely hanging on by a thread in this (deliberately) hyper-competitive economy: your goal is to make the best software you can make while surviving as best you can.

abc-1• 2 years ago

Here’s a classic example: a data type representing the state of an API request, which can be loading, loaded, or in an error state. If it has loaded, it has the expected data associated with it, and if it is an error, it has some descriptive error. A naïve implementation simply puts those all on a single data structure, with booleans for the states and optional value and error fields:

Nope, not naive at all. Not every single thing needs to be a richly expressed type system. I actually don’t like to see a million different types for some trivial state machine. It is appropriate at times and inappropriate at other times.

Not trying to sound like a grumpy old man- but juniors read things like this and then want to make a bunch of complex types to make “errors unrepresentable” when it’s totally unnecessary and complicating in many cases.

Terr_• 2 years ago

Perhaps an intermediate position be: "You can use a loose dict/map where a different subset of keys/values are in-use for different situations, but there should be a single specific discriminator key-value which is always set, one that unambiguously tells people which form is being used."

That's in contrast to logic like:

    if result.data is not None:
      # Look for successful results
    elif result.error is not None:
      # Log or handle error
    elif result.background_processing:
      # Oops, optional asynchrony
    else:
      # Must be in-progress, right?

valenterry2• 2 years ago

In what cases would that be complicating things? It usually simplifies things. The only cases that I can see where it might complicate things is if your programming language is too low level or un-ergonomic to properly express or work with sumtypes/uniontypes.

eru2• 2 years ago

Well, Java about 20 years ago would have made this kind of thing really hard.

(Or Go about 10 years ago.

I don't know if these languages got better in the meantime, I'm just reporting on my experiences from back then.)

eru2• 2 years ago

I actually don’t like to see a million different types for some trivial state machine.

It's just one type, a sum type.

klysm4• 2 years ago

I guess we have different ideas of what complexity is. I like it when the compiler doesn’t let me write incorrect code

sevensor• 2 years ago

I disagree. Constructing a data type that can be in an invalid state is packing powder in the shells for your footgun. I love a trivial state machine represented with types. Trivial code is wonderful; I have bigger fish to fry than keeping track of states represented implicitly.

That being said, there’s a real tendency to take “make illegal states unrepresentable” the wrong way. I see it used as an excuse for sheer laziness, resulting in code that takes a needlessly narrow view of what constitutes valid input and handles errors in the most graceless possible way. Just because some data isn’t a valid input, doesn’t mean dropping it on the floor is always the right thing to do.

Take handling json as an example. There are at least three layers. 1. Is it syntactically valid json? 2. Is it structurally valid? Objects have the expected fields, arrays are not scalars, and so on? 3. Is it semantically valid? Numbers are within bounds, strings are not too long? For each layer, the appropriate response is different, and for each layer there’s an appropriate representation for failure. That is: failure is a legal state, and it must be representable. Confusion about this has led to some pretty poor software of late.

tomjen3• 2 years ago

Really? You need one type “either”, which then wraps your return type or an error as appropriate, giving you compile time guarantees.

That’s not a million little classes, that is one class.

kidintech• 2 years ago

Forgettable fodder, as so many of these recent allegedly groundbreaking talks are. I don't know if it's my increased exposure to them or something else.

stonethrowaway• 2 years ago

Age combined with seeing the trite and cliche hooks to get people to read/listen/watch the usual self-promoting puffery over and over again. Sometimes it’s just the author discovering something mundane and wrapping it up as a TED talk.

chriskrycho3• 2 years ago

In some sense even the best talks are an attempt to distill things we have learned into a tight enough package that it helps people (a) think a little differently and (b) go read and learn more themselves. Sometimes that turns into self-promoting puffery, but I don’t claim to have anything particularly original to say in this talk. To the contrary: the whole point and structure of the talk is “This stuff is out there, let’s pull it together and get thinking about it!”—pointing to good work other people have done.

Now, maybe it lands that way and maybe it doesn’t, but I think that our industry could use lots of “discovering something mundane and wrapping it up as a TED talk” if it helps us to be more serious practitioners of the discipline. Most of what we need is mundane, after all! But that doesn’t mean it is unimportant, and it doesn’t mean everyone already knows it.

kreyenborgi• 2 years ago

I'm reminded of a conversation I had with someone who decades ago had worked on a spell checker for Norwegian, they had sent in a request to the "language council" requesting them to consider certain changes to the official spelling norms in order to make it all more uniform (and thus easier to write spell checkers for). Of course, software developers know better than their users. High modernism indeed, could be an example out of James Scott's book.

austin-cheney5• 2 years ago

Engineering (According to Wikipedia)

Engineering is the practice of using natural science, mathematics, and the engineering design process to solve technical problems, increase efficiency and productivity, and improve systems.

In short engineering is the ability to follow processes and measure things. Most developers I have worked with are incapable or unwilling to measure things (on any level), therefore they are incapable or unwilling to be engineers.

If you want better software set minimal, measurable, objective standards. That becomes your foundation, your law. It is as much a soft skill as a technical skill, which is explained just before halfway through the video. Software is very very bad at this and is almost entirely reliant upon compile checks to just magically set some implicit success criteria. Most developers will balk at the idea of a disciplined planned approach and just expect some abstraction to do it for them.

Once you have achieved the process you test for compliance and you measure the result. Testing for compliance isn't a manual QA effort or tiny test units. Its a total process, a certification that you are willing to bet your job and credibility upon. For software such discipline is often foreign. In other jobs like medicine, law, law enforcement, soldiering, and even truck driving getting this horribly wrong will terminate your career forever and can land you in jail or result in devastating civil suits.

Until software is willing to become an adult like those other professions this will remain a distant challenge and its a monumental drain on the resilience of those developers who try their best to achieve that disciplined intent.

spyke112• 2 years ago

Maybe it’s not the engineers that need to grow up, but the indecisive product people incapable of sticking to their principles for more than a week at a time.

That statement may be a bit much, but working in organizations unable to, well, organize around ideas leads to the state we’re in today, where most developers has to run around like headless chickens and put out fires. There’s exceptions, but from my point of view they are pretty rare.

kbolino• 2 years ago

There's definitely some interplay here.

I've worked with what I consider to be true software engineers. They cared about the craft, about the quality of what they produced, and about the process to get there. But that team was broken up because it worked too slowly relative to less caring developers, and stuck too closely to established tech instead of shiny new things. Some of the people remained, but they were now just software developers, writing code to deliver features and meet deadlines, now counting down the days to retirement.

I've also seen quality focus conflated with architecture astronauts since both take more upfront time. When product requirements are vague, there's a tendency to design a more generalized system than strictly necessary. But not all slower speeds are for the same reason. Taking some time to think out how to meet current and immediately foreseeable needs is not the same as wasting time designing for all possibilities, but a lot of people don't seem to understand the difference. Also, sometimes you do have to go through a bit of overengineering just to learn the shape and boundaries of what you're doing.

ryandrake• 2 years ago

Yes, yes, it’s “the product people.” Or “the managers.” Anyone but ourselves. If they would just stop changing requirements, we’d stop writing awful, buggy software. Not sure I really buy it. I’ve worked in software my whole life and I have never met any software engineering team who would magically have more discipline and rigor if only the managers went away. A team that cuts corners and shortchanges testing when the clock is ticking is likely also going to do that when the deadlines are more achievable. Attention to detail, measurement, thoroughness, an obsession with correctness… these are not environment specific things that come and go. They are traits of individual developers. You either have them or you don’t, and a screaming, impatient “product guy” can’t take these things out of you.

I'll admit that companies can set up processes that encourage (or discourage) rigor, and I firmly believe you can train the yolo out of software engineers. So it's not hopeless. It is very helpful to be in a company that values these things.

austin-cheney5• 2 years ago

It's actually neither. Its the software leadership. Ideally the product people should know what they want and provide that in writing, because that is what they are getting (whether or not it's what they really want). The developers/engineers are responsible for building according to the requirements and constraints provided.

Somebody has to own that. Ownership is important because that's where risk and failure live, and somebody ultimately must make the adult decisions that drives the product forward. Software leadership own the process definitions and risk acceptance while project managers own the budget and process operations.

In software this is often tiny, so its easy to gloss over and get sloppy. Consider freeway construction though which deals with a project budget that can exceed $5-20 billion, physical inventory, hundreds of people on site providing labor, and much more. There is still engineering, product delivery, a customer, and such but the liability is greater, so the planning and ownership are more disciplined. The incentives are also greater. As a result planning and modeling become more important. Injuries and defects cost money and result in halted work, so you have to account for risks and personnel in your planning. When there is no liability and limited incentives people have no real motivation to take ownership.

Other industries solve for this with some combination of licensing and broker/agent model. Licensing solves two problems. First, it sets the minimally acceptable baseline to practice and second it defines a set of ethics that become more important than employer directives.

svilen_dobrev• 2 years ago

Agreed. But further on the how-to-get-there:

Quoting works of Phillip Armour [0], circa 2000-2003:

... software is not a product but a medium for storing knowledge, and software development is not a product-producing activity, it is a knowledge-acquiring activity.

... the real job is not writing the code, or even building the system - it is acquiring the necessary knowledge to build the system... Code is simply a by-product of this activity. The problem arises when we think the code, rather than the knowledge in the code, is the product.

... When we use models and mindsets that are rigid and deterministic to manage an activity that is fluid and variable, it is not surprising that people get disappointed.

------

so, IMO, it's the nature of Software (=knowledge) that spoils the (most) practitioneers - there's no gravity (or reality as such) there, so everything is (or seems) possible. And once you learn that "easy" way.. hard to unlearn.

The software as engineering discipline needs a set of substitutes for the missing nature/reality's "laws/rules". i.e. Resiliency might be one.. All the titles of the chapters of the textbooks [1] might be candidates (correctness, maintainability/ repairability, flexibility, portability, understandability, ...). And then, depending on the domain / importance, choose smaller or bigger subset, with looser or tighter conditions.

Easier to say than do it.. Most systematically-thinking people would do it implicitly - but that as well may (or will) contradict the business goals.

Just thinking aloud..

[0] http://www.amazon.com/Laws-Software-Process-Production-Manag...

[1] https://software-engineering-book.com/

aleph_minus_one• 2 years ago

Most developers I have worked with are incapable or unwilling to measure things (on any level), therefore they are incapable or unwilling to be engineers.

You should rather consider this is a sign that software development is also partly an art, and not only an engineering discipline. Also, not all (good) developers are the same: some are more "artsy" and some are more "engineeringy".

austin-cheney5• 2 years ago

The elegance of software architecture is an art, the ability to organize things in a way that scales is multiple directions. The ability to practice, such as solving for data management, accessibility, security, performance, and such are disciplines that can be largely addressed by exceeding conformance to specified guidance. That is not a 100% cure, but it is certainly a greater than 80% cure. Due diligence is all that is expected.

My favorite example is accessibility, because the conformance document is so well written. Accessibility can be really challenging, but the WCAG definitions are so well written with step-by-step success criteria that accessibility can be achievable by anybody.

https://www.w3.org/TR/WCAG22/

klysm4• 2 years ago

If you want better software set minimal, measurable, objective standards. That becomes your foundation, your law.

This is completely intractable in practice. The things that matter cannot be readily measured. Measuring a proxy to the things that matter is fraught with peril.

I’d love to see a concrete example of what you’re thinking here because in my experience it’s not possible.

_aavaa_3• 2 years ago

It is intractable in practice since there's no outside force putting pressure to do so. Actual engineering disciplines seem to manage to have such standards, and it's in part because there are real legal and monetary liabilities at stake, unlike in software.

austin-cheney5• 2 years ago

I have heard people say this many times, but it is always wrong. Always. Absolutely everything can be measured in some way. The measures that shake out are not always the most accurate gauges of success criteria, and might even be misleading, but to make the blanket statement that things are beyond measure is like anything else is equally impossible.

Some measures are better than no measures, but if you don't trust the measures always gather additional evidence to disqualify them.

jareklupinski• 2 years ago

Most developers I have worked with are incapable or unwilling to measure things

a quick start is setting logging level to Debug

zombot• 2 years ago

a disciplined planned approach

But that would not be "agile"! Programming is mostly fashion-driven and any overlap with actual engineering is completely random.

happimess• 2 years ago

This is an hour long survey of the accumulated wisdom of a software developer who clearly cares a lot about the craft.

I listened to it this morning while doing my stoutness exercises, and there's a lot of good wisdom here. I'll probably chase down some of the books he mentions. If you're not concerned with the philosophical preamble, skip to the 5:15 mark.

brief snark: drink when he name-drops Linkedin

chriskrycho3• 2 years ago

Thanks! The books and essays are fantastic, and if I get a few people to read them, this will have been a successful talk!

Re: LinkedIn—Ha! I debated on how much to include/not-include specific references to my previous employer there, and erred on the side of “let’s keep this grounded in specific real-world experience at least a little”. I think you might get mildly tipsy, but not terribly drunk.

from-nibly• 2 years ago

The best way to do something is to not have to. Sometimes the best software is no software at all.

Seeing Like a Programmer: Resiliency, Limits, and Moral Hazards

Related Stories