You are never taught how to build quality software

https://www.florianbellmann.com/blog/never-taught-qa

by RunOrVeith • 3 years ago

592 484 3 years ago

randomdata6• 3 years ago

Is there any human activity where quality is an attribute successfully taught? In my experience, being able to produce something of quality is gained only through practice, practice, practice.

ordersofmag• 3 years ago

Good teaching largely consists of setting the learner up in situations where they can practice effectively. To pick just one example many people are taught to improve the quality of their writing. This largely consists of giving guidance on what writing to attempt and (more importantly) guidance how to reflect on the quality of the writing you've just done so you can improve.

marcosdumay4• 3 years ago

Is there any human activity where quality is an attribute successfully taught?

Every industrial practice.

On the other hand, the title just means that programming is not an industrial practice. What should be obvious to anybody that looked, but some people insist on not seeing it.

addicted• 3 years ago

Yeah and you can see other disciplines like Aviation where there are so many incredible processes to ensure learning and constant improvement.

rockemsockem2• 3 years ago

Citation needed. The key to the industrial revolution was trivializing the human work so as to take as many human errors out as possible and to systematize everything. I wouldn't call that type of process "teaching quality".

randomdata6• 3 years ago

What industrial practice isn't guided by carefully designed machines that remove the human variability?

randomdrake• 3 years ago

The arts. The further and further you go in instruction, the more it becomes about the little differences and quality. Practice always helps, but quality definitely taught and learned by many as well.

piloto_ciego2• 3 years ago

Absolutely.

I cannot fly professionally anymore due to health, but this is something we are taught in aviation and something I too find lacking from tech so far.

Like, you’re taught the standards as part of learning to fly, but as time goes on, you’re told to narrow your tolerance of what is acceptable. So if you are learning how to do steep turns, for instance, the altitude standard is +- 100’. You’re taught, “that’s the minimum, you should be trying for 50’” and then 20’, then the absolute best performance would be where you do your turn, the needle doesn’t move, and as you roll out on a calm day you fly through your wake. But the goal is “better, always better, what can I do better?” And flying is not graded on the overall performance, if you don’t do satisfactory everywhere you fail. Culturally satisfactory, isn’t, it’s the starting point.

That encourages a much more collaborative model I feel like. I’ve only worked one or two flying jobs that were not collaborative. In the outside world it sometimes feels the opposite. In flying, you truly want everyone to succeed and do well, and the company does. Even the guys I hated that I flew with, I didn’t want them to fail. If they failed, I was partially responsible for that.

It wasn’t always perfect, and I worked with some legendary assholes while I was flying, but truly, they supported me and I supported them, and if I screwed up (or they screwed up) the culture required that we found a way to minimize the future potential screwups.

You’re actually trained on what quality means in a wide variety of contexts too, and flight operations quality assurance (FOQA) is a big part of many airlines. In smaller bush operations where I worked, it is significantly more informal, but we truly had a “no fault” culture in nearly all the places I worked. It’s not perfect, but that’s the point, “ok how can we make this better?”

If someone had an idea for how to do something better, there may have been friction, but that was rare if it actually was better, and as soon as you could show how adoption was fast even at the less standardized operations I worked at.

Not saying it’s all unicorns and rainbows, but I feel like quality, and decision making, and “doing the right thing” were an integral part of the culture of aviation. “The weather is too bad and I cannot do this safely” grounds the flight, you don’t blast off into the shit (at reputable operators) to get the job done anymore (it’s not 1995), and it feels like this new industry is the opposite.

The entire concept of a “minimum viable product” is somewhat illustrative of the problem. It shouldn’t be the “minimum viable” it should be the “minimum quality product we’re willing to accept as a starting point.” But that doesn’t roll off the tongue in the same way.

We shouldn’t be striving for anything that’s the “minimum.” The minimum is only the beginning.

randomdata6• 3 years ago

But the goal is “better, always better, what can I do better?”

Is that not the case in software? The incentive to improve may not be quite as strong as in aviation (crashing software isn't quite the same as crashing airplanes), but it is still pretty strong. Life is very miserable in software when quality isn't present.

globular-toast• 3 years ago

Practice only matters if you try to produce quality. If you just practice producing crap you'll only get good at producing crap. But I suppose if someone doesn't care about quality (and these people do exist) all bets are off really.

ysofunny• 3 years ago

you cannot be taught what nobody knows how to do

or maybe, them who know how to do this are just unable to spread this knowledge... something about how they think their private secret codes are the source of their wealth

when in fact, it's merely the scheme by which they mantain an advantageous capacity to extract energy from them seeking to learn how to build quality software

Verdex8• 3 years ago

This is what I came here for.

There's an obvious comprehensibility complexity to code to anyone who has spent almost any time what so ever trying to make something happen in software. However, we've got zero academics or theory around it.

Just 'best practices' (ie a thing other people are known to do so if things go wrong we can deflect blame).

And code smells (ie the code makes your tummy feel bad. yay objective measures).

And dogma (ie "only ONE return point per function" or "TDD or criminal neglect charges").

Sure, please do something for QA because it'll be better than nothing. But we're probably a few decades of waiting for actual theoretical underpinnings that will actually allow us to make objective tradeoffs and measurements.

readthenotes1• 3 years ago

You forgot the dogma of only one entry point per function, back from the day when you could do both.

(One exit point is a pet peeve of mine since it often makes the code a lot harder to read and think about vs exit asap)

pjmlp4• 3 years ago

There is plenty of academics on it, as real engineers, those that studied Software Engineering or Informatics Engineering, instead of fake engineering titles from bootcamps, should be aware.

Usually available as optional lectures during the degree, or later as Msc and PhD subjects.

RandyRanderson• 3 years ago

I wish more ppl felt this way. What a compliment it is to oneself when I hear ppl saying "write clean code" as if they know its address and had dinner with clean code just last night.

I was thinking there should be some metric around d(code)/dt . That is, as the software is used, 'bad' code will tend to change a lot but add no functionality. 'Good' code will change little even when it's used mode.

satisfice2• 3 years ago

The scientific groundwork for excellent testing, anyway, has already been done-- but not in the realm of computer science. This is because computer scientists are singularly ill equipped to study what computer scientists do. In other words, if you want to understand testing, you have to watch testers at work, and that is social science research. CS does not take social science seriously.

An example of such research done well can be found in Exploring Science, by Klahr. The author and his colleagues look very closely at how people interact with a system and experiment with it, leading to wonderful insights about testing processes. I've incorporated those lessons into my classes on software testing, for instance.

aschearer• 3 years ago

I found these thought provoking:

The Power of 10: Rules for Developing Safety-Critical Code[1]

Assessing the Relationship between Software Assertions and Code Quality: An Empirical Investigation[2]

Cyclomatic Complexity and Basis Path Testing Study[3]

[1] https://web.eecs.umich.edu/~imarkov/10rules.pdf

[2] https://www.microsoft.com/en-us/research/wp-content/uploads/...

[3] https://ntrs.nasa.gov/api/citations/20205011566/downloads/20...

drewcoo• 3 years ago

you cannot be taught what nobody knows how to do

It's worse than that. No one can agree what "quality" means.

Mostly, the word is used as a weapon.

The pointy end of the weapon is what management pokes you with whenever anything unexpected happens. Typically they do this instead of making sure that problems do not happen (a.k.a. "management").

The weapon's handle is sometimes flourished by process gatekeepers who insist on slowing everything down and asserting their self-worth. This is not good for throughput, anyone else's mood, or eventually even for the gatekeepers.

People usually refuse to talk about quality in terms of actual business metrics because if anything unexpected happens that's not covered by the metrics, there will be fault-finding. And the fingers pointed for wrong metrics are typically pointed at middle management.

slily2• 3 years ago

I was actually thought how to build quality software (which is not limited to "having no bugs") in college, but I do not have the time or resources to apply this knowledge consistently in a corporate setting because of the pressure to deliver.

groestl6• 3 years ago

Because frankly, too much quality is not necessary, in many many cases. To know when you should or should not emphasize quality over quantity and speed, to meet a certain financial objective, is actually harder than writing quality software in the first place, I think.

slily2• 3 years ago

I agree in principle, but in my experience quality is not nearly prioritized highly enough. There is not enough understanding of quality attributes beyond the most visible ones like compute performance and stability (i.e. lack of bugs). And even for those I work on projects where people complain about lack of proper test coverage constantly, but it is impossible to dedicate time to improve that.

Xeamek6• 3 years ago

This is false. Its just that the costs of low quality code are much less obvious and harder to measure then the dev time. But the ammount of bad code just piles on itself over and over and over and we end up in a world where hardwares becomes incrementally faster while software becomes slower and slower, and more bugier. I mean, in the strict sense of the world an individual company will not pay those costs, but on a societal scale, how much time (and thus money) is wasted daily by all the people who are waiting 30secconds for windows explorer to load? If your app have millions of users, literally every additional second your app wastes multiplies to tangential numbers.

It's akin to pollution, really: Individual company making 'dirty' things won't see the consequences. But scale this mindset out and suddenly we wake up in a world when trillions of dollars are spend to counteract those effects.

teknopaul2• 3 years ago

Me too, Manchester uni was good.

I have written non-trivial systems deployed in 20 different sites that have never had a bug ever.

My best are usually second systems, focus on simplicity, standardisation and resist scope creep.

100% coverage with unit tests. 100% coverage with integration tests.

I've written a many things with zero bugs after delivery.

(and other that were never ending quality nightmares)

If I am under pressure to deliver I get strict with TDD, since no time for bug fixing.

wellpast• 3 years ago

'If we don't do it now, development efforts (and therefore also costs) will be up 15% in 4 months.'

Yeah you won't get to a point where you'll have a valid-enough metric to make this point.

I was at a startup once. The two founders said "don't write unit tests". I wasn't going to argue with them. I understood what they really meant. We've been too slow, we need to ship as fast as possible. I shipped fast and I shipped quality (ie low defects and outages). I wrote unit tests. They didn't need to know. They just needed the outcome.

The elephant in the room in all of these conversations is that you walk into any software development shop and they just don't how to ship both fast and at quality. No matter how much an organization or team tries to revisit/refactor their dev process year-to-year, they're still shipping too slow and mediocre quality.

The truth is there isn't a magic formula. It's an individual craft that gets you there. It's a team sport. And the context is different enough everywhere you go, yeah, sure, some lightweight processes might abstract across the universe but very little bang for those bucks. Far beyond any other facet of things, you really just have to have a good team with experience and the right value-system/value-delivery-focused wisdom.

Dudester230602• 3 years ago

Indeed:

"I want to start programming, what language should I learn?"

"One of these two most popular languages, where you will not even know about basic errors until you run the code!"

johngossman• 3 years ago

There are Computer Engineering programs and a few universities that really emphasize internships and hands on practice. But at many universities, the CS department came out of the Math department and is focused on theory. Chemistry isn't Chemical Engineering either. I think that's okay. University isn't just a trade school--the idea behind almost any degree is to train the mind and demonstrate an ability to master complex material.

timeagain• 3 years ago

Yeah but at those internships you aren’t taught how to build quality software, just how to ship a SPA that connects to an API in 15 weeks (or you’re not hired).

It is a good peek into the professional software world though!

bluGill16• 3 years ago

Before you can write quality software you need to be able to write large software. Most interns I see are learning how to work on non-trivial programs as this is their first chance to see something non-trivial. Then they get a "real job" and are shoved into extremely large programs.

Writing a thousand lines of bug free code isn't that hard, so the need for QA practices won't be apparent. Then you get the ten thousand line intern project and discover things are not always that easy. Then we throw you into a multi-million line project and good luck: you need a lot of that QA stuff to make it work.

bluGill16• 3 years ago

What society needs is a mix of trade school a traditional university. If a university is not providing both they are failing everyone. (except the straw-man rich kid who will inherit a lot of money but not be expected to either also inherit/run a company or pass the money onto their kids - this is something that happens in story books but doesn't seem to be real world where the rich give their kids lots of advantages but eventually expect them to take over and run the family business)

A pure university education without considering is this degree useful in the real world is a disservice to education. However a pure trade school education that teaches how to do something without understanding is not useful (I don't think any trade school is that pure: they tell you to ignore hard stuff but generally give you deep understanding of some important things)

duped4• 3 years ago

If a university is not providing both they are failing everyone.

Why?

A pure university education without considering is this degree useful in the real world is a disservice to education.

I think this line of thinking is a much bigger disservice to higher education. It was very tiresome as an undergraduate to be surrounded by people that thought this way - and detrimental to everyone's education.

"I'll never use this knowledge" is the single worst thing you can say as a student, and it needs to be beaten out of undergrads' heads. Not encouraged.

KeplerBoy3• 3 years ago

It will be necessary to deliver software without bugs in time.

Seems like a pretty bad premise to start an article on quality software. If you believe you can ship bug free code, it's time to switch careers.

FalconSensei3• 3 years ago

If you believe you can ship bug free code, it's time to switch careers.

Unfortunately, you are correct. Shipping in time and bug free are inversely proportional, and in a world were usually it's hard to argue with PMs for more time to have better testing, or paying tech debt... it's just a reality

colinmorelli3• 3 years ago

An infinite amount of time would not necessarily yield zero bugs.

But more importantly, once you've fixed the "show-stopping bugs," putting the software in front of customers is probably the best next step, as even if it's bug-free, that doesn't mean it solves the problem well.

nradov• 3 years ago

That's only true up to a point. There are some quality assurance and control activities that are essentially "free" in that they actually allow for shipping faster by preventing rework. But that requires a high level of team maturity and process discipline, so some teams are simply incapable of doing it. And even in ideal circumstances it's impossible to ship defect free software (plus the endless discussions over whether particular issues are bugs or enhancement requests).

d0gsg0w00f• 3 years ago

Like the age old builder trope.

"Cheap. Fast. Good. Pick two."

OnlyMortal2• 3 years ago

Correct. As I like to tell my team, if I’ve typed something I’ve caused a bug. It’s all about risk.

I assume I’m not infallible.

Writing some unit tests, C++ and mocking in my case, give both the team and myself some confidence I’ve not made things worse.

I’m the most experienced dev in the department.

reactordev3• 3 years ago

This kind of wisdom only comes from experience I think. Either that or higher order think. Like the article says, most of the time testing/tdd/qa is bolt on after-the-fact. Or a big push at the end with "QA Sprints" (are you sprinting or are you examining? what exactly is a QA sprint? I know what it is).

Once you get beyond "I wrote a function" and "I tested a function" and even still "I tested a function that was called by a function over the wire", you will come to a realization that no matter how edgy your edge cases, no matter how thorough your QA, there will always - ALWAYS be 0-day "undefined behavior" in certain configurations. On certain hardware. On certain kernels. It's an assurance that I guarantee that I'm almost positive it's bug free, since it passed tests, it passed human eyes, and it passed review - fingers crossed.

chopsuey5540• 3 years ago

You might be correct today but that’s a pretty sad state of affairs, don’t you think we can do better? Most other engineering domains can deliver projects without bugs, with various definitions of “bug” of course

KeplerBoy3• 3 years ago

I'm not sure about that. Which engineering domain do you have in mind?

Maybe show-stopping bugs are somewhat unique to software engineering, but all somewhat-complex products are flawed to some extent imho.

It might be an unergonomic handle on a frying pan, furniture that visibly warps under the slightest load (looking at ikea shelfing) or the lettering coming off the frequently used buttons on a coffee machine.

ElectricalUnion2• 3 years ago

To be an engineer is to know the expected system requirements and build a product that is extremely optimized for the system requirements.

There's a saying that I think fits very well here: "Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands."

You don't want a bridge to cost 50 years and quadrillions of dollars to build, you want a cheap bridge safe for the next 50 years done in 2 years.

I would not call the resulting bridge "bug free", of course.

sanderjd11• 3 years ago

Their stuff has bugs too.

imiric• 3 years ago

We can certainly do better, but it takes a _lot_ of time, effort, care and discipline; something most teams don't have, and most projects can't afford.

Bugs arise from the inherent complexity introduced by writing code, and our inability to foresee all the logical paths a machine can take. If we're disciplined, we write more code to test the scenarios we can think of, which is an extremely arduous process, that even with the most thorough testing practices (e.g. SQLite) still can't produce failproof software. This is partly because, while we can control our own software to a certain degree, we have no control over the inputs it receives and all of its combinations, nor over the environment it runs in, which is also built by other humans, and has its own set of bugs. The fact modern computing works at all is nothing short of remarkable.

But I'm optimistic about AI doing much better. Not the general pattern matching models we use today, though these are still helpful with chore tasks, as a reference tool, and will continue to improve in ways that help us write less bugs, with less effort. But eventually, AI will be able to evaluate all possible branches of execution, and arrive at the solution with the least probability of failing. Once it also controls the environment the software runs in and its inputs, it will be able to modify all of these variables to produce the desired outcome. There won't be a large demand for human-written software once this happens. We might even ban software by humans from being used in critical environments, just like we'll ban humans from driving cars on public roads. We'll probably find the lower quality and bugs amusing and charming, so there will be some demand for this type of software, but it will be written by hobbyists and enjoyed by a niche audience.

fidotron8• 3 years ago

It is sad that people on here would believe this and that for whole platforms it is actually true, however, it absolutely is not universally true and the proof is all around us.

sanderjd11• 3 years ago

What proof is all around us?

jasondigitized• 3 years ago

It's perfectly acceptable to let bugs escape into production if those "cost" of fixing that bug higher than the "cost" to the user experience / job to be done. A bug that takes a week to fix that will only be encountered by a small amount of users in a small number of obscure scenarios may not need to be fixed.

AlotOfReading2• 3 years ago

This presupposes that you know what/where bugs will be found and how they'll impact future users. In my experience knowing either of these is very rare at the point where you're "building quality".

billti• 3 years ago

I think a common error is taking this view in isolation on each bug.

Fact is, if you ship enough 'low probability' bugs in your product, your probabilities still add up to a point where many customers are going to hit several of them.

I've used plenty of products that suffer from 'death by a thousands cuts'. Are the bugs I hit "ship blockers"? No. Do I hit enough of them that the product sucks and I don't want to use it? Absolutely.

Dalewyn3• 3 years ago

A saying that I once heard and appreciate goes like this:

"A programmer who releases buggy software and fixes them is better than a programmer who always releases perfect software in one shot, because the latter doesn't know how to fix bugs."

Perhaps similar to the saying that a good driver will miss a turn, but a bad driver never misses one.

MaxBarraclough6• 3 years ago

That's backward. A successful software development methodology will tend to catch bugs early in the development pipeline.

The doesn't know how to fix bugs idea seems pretty silly.

codegeek2• 3 years ago

Yep. If you have written production grade software at real companies, you know that the moment you make that new commit (even if 1 liner change), you are now ready to accept that it could break something. yes you can do your unit tests, integration test, User Acceptance Tests and what not. But every code change = new possible bug that you may not be able to catch until it occurs to a customer.

Whenever I hear a developer say "I never ship buggy code", I am always cautious to dig in more and understand what they mean by that.

Gare• 3 years ago

How about a formal proof? :)

I jest, but that should be the gold standard for anything life-critical and good to have for mission-critical software. Alas, we're not there yet.

wvenable• 3 years ago

It's always amazing when I get a bug report from a product that's been running bug free in production for years with minimal changes but some user did some combination of things that had never been done and it blows up.

Usually it's something extremely simple to fix too.

datadeft• 3 years ago

Spicy take on engineering. Why do we accept this for software when do not accept the same in other engineering domains?

alexchamberlain• 3 years ago

Most engineering domains expect failure; the fail safes, checklists etc prevent it causing real damage.

Gud3• 3 years ago

Trust me when I say this: even "other" engineering domains have to do patches.

The difference is that software can be used before it is fully ready, and it makes sense to do so. No one can really use a 90% finished power plant, but software at 95% capacity is still usually "good enough"

benwilson-512• 3 years ago

My wife works as an acoustical consultant at a global construction firm. The things you hear about factories, offices, and even hospitals is wild. Don’t get me wrong the construction world works very hard to avoid issues but I think we in software tend to hold other engineering disciplines up on a pedestal that doesn’t quite match the messiness of reality.

pixl972• 3 years ago

when do not accept the same in other engineering domains?

No, you just complain that your taxes are being used to build expensive roads and bridges. Or you think airplanes are far too expensive. Or that new cars are insanely expensive.

There are cost trade offs. In general, better quality more expense.

Also in software there is not an excessive amount of software engineers in relation to demand for software. So SWEs can get paid a lot to go build crappy software.

ska6• 3 years ago

There are a few aspects. One is that we don't understand the fundamentals of software as well as the underpinnings of other engineering disciplines.

More importantly though, for the most part we choose not to do engineering. By which I mean this - we know how to do this better, and we apply those techniques in areas where the consequences of failure are high. Aerospace, medical devices, etc.

It differs a bit industry to industry, but overall the lessons are the same. On the whole it a) looks a lot more like "typical" engineering than most software development and b) it is more expensive and slower.

Overall, we seem to have collectively decided we are fine with flakier software that delivers new and more complex things faster, except where errors tend to kill people or expensive machines without intending to.

The other contributing thing is it's typically vastly cheaper to fix software errors after the fact than, say, bridges.

BeefyMcGhee• 3 years ago

Because other engineering domains are "actual" engineering domains. They didn't just co-opt the word to have fancier sounding job titles.

rockemsockem2• 3 years ago

We accept this in all fields of engineering. Everything is "good enough" and the seems to work reasonably well. You should remember this next time you hear about car recalls, maintenance work on bridges, or when some component in your laptop flakes out.

digging5• 3 years ago

In addition to the other answers, there is the perennial and depressing one: Software bugs haven't killed enough people in a suitably visible/dramatic way to be regulated that heavily.

ponector9• 3 years ago

Imagine same approache in other domains:

Team are flying the airplane, the se time rebuild it to the zeppelin, testing new engines inflight.

Or construction. Let's build apartment block, but for few apartments we will test new materials, new layout, etc. Once there are walls of the first apartments we will let people live there. We will build how we can, according to the napkin plan. In the end we will put all tenants in and stress test strength of the structures. Or one day people return home and their apartments have totally different design and layout because someone from the HOA decided so to get a promotion.

rocqua3• 3 years ago

I mean, bridges collapse. That hasn't meant we gave up on engineering bridges. Point being, we have some risk tolerance, even for civil engineering.

Now we don't accept an engineer saying, "this bridge will probably collapse without warning", which we do accept with software. So there is a difference.

kaba06• 3 years ago

Because complexity is boundless and in software it has no cost.

Building a house will have a restrictive initial budget for complexity, you don’t have enough in that budget for rotating floors, or an elevator that is catapulted to the correct floor, etc. These would cost both at engineering time and implementation time a huge amount. Less complexity is easier to analyze.

In case of software, complexity has negligible cost, relative to physical systems. You can increase it ad infinity — but proving it (the whole stack - from the hardware-OS-userspace software) correct is likely impossible with even the whole of mathematics, in certain cases.

fivre• 3 years ago

it will be necessary to deliver software without bugs that could have reasonably been avoided in time

ive had this sentiment thrown at me too often by peak move fast and break things types. it's too often a cudgel to dispense with all QA in favor of more new feature development. shipping shit that has the same pattern of flaws youve encountered in the past when youve been shown ways to catch them early but couldnt be bothered isnt accepting that you cant catch everything, it's creating a negative externality.

you usually can make it someone else's problem and abscond with the profits despite, but that doesn't mean you should

HeyLaughingBoy3• 3 years ago

Nothing is bug free.

Not buildings, not bridges, not cars, not airplanes, not software. There are mistakes in every field of engineering and the best we can hope for is to minimize them as much as possible.

Engineering is knowing (among other things) how to categorize the potential mistakes, develop procedures to reduce the chance of them being made and in the case that some slip through (and they will), estimate their impact and decide when you're "good enough."

Software engineering is no different.

rocqua3• 3 years ago

I think with formal analysis, whole bug classes can be eliminated. Add to that a rigorous programming style, and 'bug-free' code is within reach. There will remain bugss that make it through, but they will be rare, and will need a chain of mistakes.

Currently ways of coding to this kind of standard exist. But they are stupid. It involves things like no dynamic memory allocation, only fixed length for-loops, and other very strict rules. These are used in aerospace, where bugs are rather costly and rare.

kaba06• 3 years ago

What seems to yield much better results is to have the program be built by two separate teams to a standard. Then both programs can be run simultaneously, checking each others output — I believe something like this is actually used in the aerospace industry.

sublimefire• 3 years ago

Reminds me of a story about an engineer who was participating in a meeting with managers and partners. Manager was speaking of his team and how they will deliver software. Then, he asked the engineer to assure that the software will be bug-free. To this the engineer responded by saying he cannot guarantee there will be no bugs. The manager went nuts and started screaming.

Engineers cannot be responsible for all the vertical stack and the components which were built by others. If somebody claims it is bug free then they have not enough experience. Pretty much anything can fail, we just need to test as many possible cases as possible with a variety of available tools to reduce the chances of bugs.

throwaway_08932• 3 years ago

You may not be taught to build it, but everyone sure seems to know how to argue it.

0xdeadbeefbabe• 3 years ago

Even if this teaching quality thing existed, everyone would still argue about it.

teknopaul2• 3 years ago

Author needs to change shop

Where I work is 50% qa effort at least, 50% by budgets, and devs do much >50% automated testing as part of what we call development.

Anything less and projects take longer, because bugs found later cost so much more to fix.

bluGill16• 3 years ago

When I first started in the 1990s I was told testing was 60% of the product budget and traditional writing code 20%. The remaining 20% was architecture and other design work. We didn't have unit test frameworks, but we did spend a lot of time writing throw away test fixtures (which was already generalizing into test frameworks by great developers and kicking off the unit test revolution)

warkanlock• 3 years ago

Making good (!perfect) software is a function of three constraints: knowledge, economic resources, and time.

You can mix those three together and produce a desired output, but don't expect perfection, perfect software only appears when the three variables tend to infinity

revskill• 3 years ago

Yes, know how to build quality software is the breed and butter of a software engineer, why sharing it ?

What you read on the internet is mostly ... bullshits.

elif• 3 years ago

I dunno about this authors CS program but at GT we had significant coursework related to all aspects of SDLC (including unit and acceptance testing), business case value, etc.

continuational• 3 years ago

The process of building quality applications starts long before any code is written.

You need to understand the domain, you have to design something that solves an actual problem or delivers tangible value to someone, you need a holistic approach to user experience.

withinboredom5• 3 years ago

Even if you understand the domain, you can build the wrong thing, or just simply not know something you don't know; thus you can't even learn it.

I've seen this too many times to count.

drivers99• 3 years ago

In addition, at least in my studies, there was a semester about project management approaches and scrum. All of which is great, but QA is missing completely.

Just an anecdote, but, we had a "Software Development" class like this in CS (I took it in the '90s) and even though it followed a waterfall development model[0] and we used Gantt charts, QA (testing) was a big part of it and 1 of our 4 team members (or maybe 2 of 4 worked on it together) was primarily responsible for it. (I wrote the parser and the made diagrams/documentation for the parser.)

The description (in an old catalog[1]) is:

Software specification, design, testing, maintenance, documentation; informal proof methods; team implementation of a large project.

Turns out I didn't need to look up the old catalog because the description is exactly the same still! Except it's CS 371 now, and the longer "Learning Outcomes" for the course has some newer stuff (agile and version control) but otherwise is all the same things I learned at the time.

[0] https://en.wikipedia.org/wiki/Waterfall_model

[1] https://nmsu.contentdm.oclc.org/digital/collection/catalogs/... (C S 372 in the lower right corner)

btown• 3 years ago

While not a panacea, visual/snapshot testing tools like Cypress + Percy that perform clicks and take screenshots can be tremendously helpful to ensure that building in a programmatic QA plan is a predictable part of the scope of what is coded.

And the good thing about snapshots is that they provide a visual way to communicate a preview to stakeholders of what may change, both for the currently-being-developed workflow as well as to prevent visual regressions elsewhere - so they're inherently more easily justifiable than "we're spending time on testing that you'll never see."

The article is correct that treating QA as the last phase of the project, and a disposable phase at that, is a recipe for disaster. But if you make it an ongoing part of a project as individual stories are burned down, and rebrand it as part of a modern development workflow, then it's an entirely different conversation.

eftychis• 3 years ago

As sibling comment `@wellpast` commented, but to extend it with my point of view.

We can roughly say we can have three out of: (high) quality, (low) time, (low) communication complexity, and (low) money. (time is a dependent here.)

People are trying to apply factory processes and structures to a team sport, an engineering discipline. You do not teach or build a basketball team by breaking down each attack phase into steps and checkmarks.

You try to minimize communication and make the team work as one. It is a team and individual building, not a process building exercise. You make a plan, and follow the Moltke's the Elder conclusion:

"no plan of operations extends with any certainty beyond the first contact with the main hostile force."

(Or paraphrased as you have heard: No plan survives contact with the enemy.)

All (types of) Engineers know this. But software engineering is "special." And it is not a "move fast and break things issue." That is part of all engineering or team playing too.

It is the type of business mentality, that because a plan did not go exactly as expected we need to add more process. Whatever that process may be. Because if "I as a manager add a process, then the next failed plan, I am covered, and I will blame the individuals."

Process has a place to ensure things happen in a legal and moral framework. And minimize adverse circumstances -- e.g. we bet all the hedge fund money accidentally when running tests.

Process is used differently in most startups and corporations with not the team in mind.

swader999• 3 years ago

The construction metaphor is a bad analogy. The compiler does the construction, dev teams do iterative design, ideally with frequent feedback and adjustment.

Do you ever yell at a traditional architect and ask them when it's going to be done? It's always when the client is happy or makes their mind up about it. A lot of dev is like this.

tobiasSoftware• 3 years ago

Honestly I think the root problem is that universities have a degree in computer science, whereas what most people want is to learn to build computer software.

The two overlap most of the time in subtle ways where the science gives an important foundation, such as learning Big O notation and low level memory concepts where exposure helps. I've personally seen this with a smart coworker who didn't go through university and is great at programming but I'll catch him on certain topics such as when he didn't know what sets and maps were and when he tries to sleep a second instead of properly wait on an event.

However, the differences between computer science and building software are problematic. Watching my wife go through university, she's had to struggle with insanely hard tasks that will not help her at all with software, such as learning Assembly and building circuits. The latest example is the class where she's learning functional programming is not actually teaching it to her. Instead, they combined it with how to build a programming language, and so instead of giving her toy problems to teach the language she is having to take complex code she doesn't understand well that generates an entirely different programming language and do things like change the associativity of the generated language. In the end, she feels like she's learned nothing in that class, despite it being her first experience with functional programming.

On the flip side are the things that are necessary for software that aren't taught in university, like QA. For me personally, back when I was in university a decade ago I never learned about version control and thought it was just for back up. Similarly, I never learned databases or web, as the required classes were instead focused on low level concepts as Assembly and hardware. My wife is at least learning these things, but even then they often seem taught badly. For example, when they tried to teach her QA, instead of hardcoded unit tests, they made her give random inputs and check to make sure the output was correct. Of course, checking the output can only be done by rewriting all of your code in the testing files, and if there's a bug in your code it'll just get copied, so that kind of defeats the purpose. Even when the assignments are relevant there is often no teaching on them. For example, her first ever web code was a project where they told her to hook up 6 different technologies that they had not gone over in class, with only the line "I hope you've learned some of these technologies already".

aidenn02• 3 years ago

I wonder how much of the lack of QA is rational. That is to say, for most projects does shipping with lots of somewhat hard-to-find bugs actually hurt the bottom line?

For some classes of bugs it can (e.g. if the software is so bad as to open you to a class-action lawsuit; in b2b software bugs that put you in breech of contract), but for many classes of consumer software, it's not clear to me that shipping software that works better is rewarded. Picking not-too-buggy software ahead of time is hard, people are slow to switch (even when the people encountering the bugs are the people selecting the software, which is often not the case), and people are good at subconsciously avoiding triggering bugs.

nikhil896• 3 years ago

It starts mattering more for consumer software when you reach mass scale. Somewhat hard-to-find bugs at the scale of hundreds of millions of users (like a social media company), turn into bugs faced by hundreds of thousands of users.

But at that scale (in my experience), QA is up front and center and is typically a core pillar of engineering orgs.

aidenn02• 3 years ago

It starts mattering more for consumer software when you reach mass scale. Somewhat hard-to-find bugs at the scale of hundreds of millions of users (like a social media company), turn into bugs faced by hundreds of thousands of users.

From a cynical point of view, if those hundreds of thousands of users will use your product despite the bugs, does it matter?

jupp0r3• 3 years ago

1. The premise that college teaches you how to build software in industry is a pretty wild claim.

2. Is this article from the 90s where we ship software on CDs or floppy disks? In today's world where the concept of a "release" is often blurred by continuous delivery pipelines (and this is considered a good practice), having a quality insurance department manually assuring that no bugs are in that release seems downright archaic.

bluGill16• 3 years ago

Not everyone is writing a webapp where you can roll out upgrades anytime CI passes, or a phone app that you can upgrade every week. some of us work on code that will be shipped in a device that is not easy to upgrade.

jupp0r3• 3 years ago

Not all software that is frequently updated is a web app. Ask Tesla, Apple, Sonos, Garmin, ...

Anything connected to a network could be released frequently if people wanted to. Not everything is connected to a network though.

ponector9• 3 years ago

And then you remember stories like that about Boeing. How their 737 Max has been tested by cheap contractor from India.

There are tons of software without continuous delivery.

jupp0r3• 3 years ago

Absolutely, and industries like medical devices and aviation have extremely strict regulation and procedures regarding the testing of software. The article not mentioning any of those made me conclude that author is referring to regular software.

withinboredom5• 3 years ago

First, define quality software. I'll wait.

shrimp_emoji• 3 years ago

Written in Rust

ragnot3• 3 years ago

I'll take a stab. Quality software is software that is testable, able to adapt to new features and is architected to match the current organizational structure of the software team so that communication and dependencies don't have an impedance mismatch.

withinboredom5• 3 years ago

Now the hard questions:

- why is testable software higher quality? Does it add value to the software? I'd venture that untestable software has the same value (if not more) than testable software (due to time-to-market). You can write software that is 'obviously correct' and "high quality" at the same time, without any tests.

- Why does software that can adapt to new features increase the quality? If that is the case, we must argue that WordPress is extremely high-quality software. Or SAP.

- How does architecture influence quality? If that is the case, then there isn't any need for different architectural styles since there should be "one true style" that has the best quality software.

NBJack• 3 years ago

Good start, but too broad and open for interpretation.

- Who gets to define testability?

- I want to add a coffee maker to my crash test dummy; is the lack of room for the filter and water tank a sign of a bad design? Or not flexible enough for my feature?

- (cue meme) "You guys have organizational structure?"

- Who gets to claim the impedence mismatch? What are those consequences? Wait, where are the dependencies defined again outside of the software?

speedgoose3• 3 years ago

https://en.m.wikipedia.org/wiki/Software_quality

It’s not that crazy.

poisonborz• 3 years ago

My entry: one that is easy to refactor regardless of code size.

If this is given, every other metric, like features, bugs, performance is just a linear dependence on development resources (maybe except documentation, but that is kinda an externality).

ahoka• 3 years ago

Easy! Software that fulfills its requirements the cheapest.

But how do you define the requirements is the real question (and problem)…

oopsthrowpass• 3 years ago

Yeah the dimensions discussed in this article is somewhat advanced stuff and comes from experience, DRY is relatively basic concept and easy to grasp and unfortunately mid level engineer do really dangerous stuff in the name of DRY and horrible abstractions get created that break down and create a horrible mess when the requirements change.

In my experience it is generally wise to avoid abstractions and copy/paste things a couple of times, once the code base matures good abstractions will be more obvious. Even then it's good to think about future changes, will these 2 things want to evolve separately in the future? If the answer is YES, then maybe coupling them is not a great idea. I think there was a really good Kent Beck talk about coupling vs cohesion somewhere.

Another thing to think about is breaking things, if changes are local to one single endpoint then any changes there can only break that endpoint, edge cases and scenarios to consider are only relevant to that endpoint. When changes to a core abstraction are required then hundreds of use cases/edge cases need to be considered - why are we creating so many core abstractions in our systems in the name of DRY?

I've also found that the more moving parts you add the harder a system becomes to learn, the S in SOLID is probably to blame for that. The only single responsibility principle is useful for is unit tests (easier to mock), but many times harder to understand. If the actual functionality is not local to the file things become ungreppable via code search, understanding the entire system requires an IDE and jumping around to each and every beautiful lpad() implementation and trying to piece what is happening one 3 line function at a time.

Then there is also layering to consider, if 2 pieces of code look somewhat similar but belong to different layers (example controller and DAO layer, then also care must be taken to not make an abstraction that couples these together, or to couple 2 unrelated modules together that could otherwise have their own life cycle).

These are just some aspects I could think of that I think about when creating abstractions, but somehow I see engineers focus too much on DRY. Maybe they got burned so bad some time in the career by forgetting to change something in 2 places?

prosaic-hacker• 3 years ago

The whole article has a straw-man feel to it. It is not the senior developers responsibility to create a proper QA policy for the project. It is good that the lead should know how to implement good QA processes.

HOWEVER, the real culprit are the MANAGERS.

After 50+ years of skanky software development policy aimed at low balling cost its time to blame the right people. No amount of cajoling, "business/budget" speak manipulation is going to fix a fundamental flaw in how managers at that level are trained and behave.

We have stop being apologists for mistakes not of our making. If using QA techniques would increase the managers bonus then we will see it being used. If all it does is make better software then this post will be rewritten in 50 years and would still be relevant.

winddude• 3 years ago

I've certainly be frustrated at others for shitty software. We're definitely taught what bad code is.

heurist• 3 years ago

Half the places I have worked had no desire for quality software..

natbennett• 3 years ago

Berkeley has a class that teaches TDD and other XP practices. (Or at least used to.) Pivotal used to recruit a lot of new grads from Berkeley for that reason.

IME, “QA” doesn’t really correlate with quality software, nor is there really a time vs. quality trade off. Bad software is often also delivered poorly, and high quality software can be delivered quickly.

titzer5• 3 years ago

We do teach these things, they are just not core CS topics, but rather in other areas, relegated to electives like a software engineering course. At CMU we have entire Master's program for software engineering and an entire PhD program (in my department). We teach exactly the kinds of things the blog post is about, and more. Software Engineering is a whole field, a whole discipline.

I get that this is a blog post and needfully short, but yes, there are courses that teach these skills. There's a big disconnect between CS and SE in general, but it's not as bad as "no one teaches how to build quality software". We do work on this.

__loam2• 3 years ago

I took one of these kinds of classes in my masters program this year. They were totally obsessed with UML. It would be nice if these classes could move beyond dogma that is decades old.

crysin• 3 years ago

Got my MS in SE at DePaul University in Chicago. Wrote more UMLs for those classes than I’ve done in 10 years of actual software development.

titzer5• 3 years ago

CMU constantly reevaluates its MSE program with input from many different angles. I've participated here and I think we're trying hard to balance important foundational knowledge with practical skills of the day. I don't think we over-emphasize UML or any one particular silver bullet in our program.

hinkley5• 3 years ago

Wait. This year?

I haven’t touched UML for ten years.

petsfed2• 3 years ago

What would be better? Change tools every 3-5 years like the industry does, so by the time any given instructor actually has a grasp on a particular tool or paradigm, its already obsolete (or at least fallen out of fashion) too?

I'm no fan of UML, but the exercise is to teach students how to plan, how to express that plan, and how to reason about other people's plans. The students will certainly draw a lot of flow diagrams in their careers, and will almost certainly deal with fussy micromanagers who demand their flow diagrams adhere to some arbitrary schema that has only limited impact on the actual quality of their work or documentation.

UML is complete, at least.

speedgoose3• 3 years ago

Yes, I learned these things as a computer science student in an engineering school. It wasn’t perfect but a good introduction.

bakul2• 3 years ago

Do SE classes teach debugging skills? I hope they do. So many times I have seen people try random things rather than follow a systematic approach.

hinkley5• 3 years ago

I worked with programmers around my junior year and some of them were in classes I was in. I thought they were all playing one-upsmanship when I heard how little time they were spending on homework. 90 minutes, sometimes an hour.

I was a lot faster than my roommate, and after I turned in my homework I’d help him debug (not solve) his. Then I was helping other people. They really did not get debugging. Definitely felt like a missing class. But it helped me out with mentoring later on. When giving people the answer can get you expelled, you have to get pretty good at asking leading questions.

Then I got a real job, and within a semester I was down below 2 hours. We just needed more practice, and lots of it.

trealira• 3 years ago

I'm not sure if software engineering classes in particular do, but at my university, they teach C++ in the second required course, and they teach you about using GDB and Valgrind on Linux there. They don't explicitly teach you about systematically debugging, though, beyond knowing how to use those two programs.

manicennui2• 3 years ago

Trying random things seems to be how a large number of professional software engineers do their jobs. Stack Overflow and now CodeGPT seem to contribute to this.

zogrodea2• 3 years ago

What do you mean by people trying random things? I think that approach (if I understand the term correctly) is more or less what debugging is as a form of scientific investigation.

If you observe a car mechanic trying to find the problem with a car, he would go like: "is this pin faulty? No. Is the combustion engine faulty? No. Are the pedals faulty? Yes." where the mechanic starts with some assumptions and disproves them by testing each of those assumptions until (hopefully) the mechanic finds the cause of the fault and is able to fix it. Similar types of investigations are important to how natural science is done.

So it would be helpful if you can clarify your intended meaning a bit more. Maybe I or someone else would learn from it.

GuB-42• 3 years ago

But debugging is about "trying out random things". You can call it a Monte-Carlo tree search is you want to sound smart.

And I don't feel is not something that is worth teaching in universities, because it is 90% experience and for me, the point of universities is not to replace experience, just give enough to students so that they are not completely clueless for their first job, the rest will come naturally.

What universities can teach you are the tools that you can use for debugging: debuggers, logging, static and dynamic analyzers, etc..., different classes of bugs: memory errors, the heap, the stack, injection, race conditions, etc..., and testing: branch and line coverage, mocking, fuzzing, etc... How to apply the techniques is down to experience.

In fact, that's what I find most disappointing is not juniors programmers struggling with debugging, this is normal, you need experience to debug efficiently and juniors don't have enough yet. The problem is when seniors are missing entire classes of tools and techniques, as in, they don't even know they exist.

sam0x17• 3 years ago

Worth mentioning that SE isn't even a thing for the most part at non STEM-specific schools and/or outside very large colleges/universities

AnimalMuppet2• 3 years ago

Here's the thing, though: Of a CS graduating class, 90% of them will work as software engineers, not as computer scientists. (All numbers made up, but I think they're about right.)

We don't need these things to be electives. We don't need them to be a master's program. We need an undergraduate software engineering program, and we need 90% of the people in CS to switch to that program instead.

titzer5• 3 years ago

I agree with you! It's hard to change curricula because there are so many competing interests. CS is an evolving field and things like machine learning have burst onto the stage, clamoring for attention. There is also an age-old debate about whether CS departments are trade schools, math departments, or science. Personally I think software engineering skills are paramount for 90% of graduates. How do we fit this into a full curriculum? What gets the axe? Unclear.

manicennui2• 3 years ago

A large number of CS programs are already glorified Java training courses.

hardwaregeek2• 3 years ago

There's a massive gap between taught at CMU and taught at most universities. And even if it is taught, it's usually outdated or focused on very literal stuff like how to write web applications. I'd have killed for a class that actually focuses on implementation, on teamwork, on building complicated systems.

sasaf5• 3 years ago

+1 for teamwork... I wished there was an established field of study for teamwork in software. If you are impressed with 10x devs, imagine 10x teams!

ricw• 3 years ago

That was a dedicated software engineering course I took at university. Teams of 5. Had to put the soft eng theory into practice. And if’s not CMU.

heelix• 3 years ago

I've wished that students would be required to hand their semester long project to the person next to them each week.

jeffreygoesto• 3 years ago

And then there was "Twenty dirty tricks to train software engineers" which I found even more suitable as preparation for working in the industry.

skeeter20205• 3 years ago

This just isn't realistic in an general CS undergrad though. Students don't have the time or foundations to build a realistic complicated system, and the institutions don't have skills or currency to teach them how. You complain about content being out of date, but the foundations of quality software are not by and large technical nor go out of style. The implementations sure do.

What you're asking for is where vocational (2 year) programs really shine, and one of the few places where I've seen bootcamp graduates have an advantage. Unfortunately both then lack some really valuable underpinnings, example: relational algebra. It seems that once again there is no silver bullet for replacing time and experience.

hinkley5• 3 years ago

If my friends hadn’t had such vividly bad experiences with the compiler class, I might not have taken the distributed computing class that was one of the other options to fulfill that category.

It’s not the most defining class of my undergrad years, but it was pretty damned close.

The fact that most people designing systems don’t know this material inspires a mix of anger and existential dread.

titzer5• 3 years ago

If you're specifically referring to CMU's compilers course, feel free to follow up with me offline.

jaybrendansmith• 3 years ago

After seeing the same mistakes made over and over again I can't help but agree. This is how one builds enterprise software now, and it is poorly understood by most developers, although that is starting to change. If I were designing a college curriculum, required courses would include all of the normal CS stuff but also software engineering, distributed computing, design patterns, web application fundamentals, compiler design, databases, DevOps/Cloud, testing fundamentals, UX design, security basics, data science, IT project/process management, essentials of SOA, product mgt and requirements. Of course, it's been so long since I went to college, none of these things existed back in the day, so perhaps modern curriculum has all of these now!

3abiton• 3 years ago

What does research in this field look like?

stephendause3• 3 years ago

So I have not done research any software engineering field and have not read all that much either. One example that comes to mind from one of my courses that I took in software engineering is research around mutation-based testing. That form of testing is where you generate random variants of your test by doing things like deleting a statement, adding a statement, changing a less than sign to a greater than sign, etc. Then you check to see that at least one of your tests fails for each variant. If it does not, you either add a test or mark that variant as being effectively the same program. I forget what the term is for it. At any rate, I think there is still research being done on this topic, for example how to effectively generate programs that do not generate as many functionally identical programs. Software testing in general is a big part of software engineering, and I think there is still a fair amount of research that could be done about it.

In my opinion, the intersection of cognitive psychology and software engineering is also ripe for a lot of research. I feel like we as software engineers have a lot of intuitions about how to be productive, but I would love to see various things that could indicate productivity measured in a controlled lab setting.

titzer5• 3 years ago

Testing, fuzzing, and verification are all pretty hot SE topics now, at least in my department at CMU. All of those have some element of program analysis, both static and dynamic, so there's potentially deep PL stuff, which is core CS. There's aspects to SE that are not just that, of course, which are studying social processes, management styles and practices, software architectures, design patterns, API design, etc.

gmadsen• 3 years ago

I would assume something like this? https://dspace.mit.edu/handle/1721.1/79551

doubloon• 3 years ago

i feel like the Toyota Production System could be applied, I know they used some of it at Amazon but in general I don't hear much about it in 'software spaces'. It's been studied a huge amount in quality theory but... seems like theres not a lot of crossover between manufacturing and software.

like the idea of a Poke Yoke could cover all manner of automated analysis software tools. from the -Wall in gcc to, Rust borrow checker, basically just Poke Yoke.

fudged71• 3 years ago

I chose software engineering. 3 years into the program the head of the department made a speech at an event to the effect of "Software hasn't changed in the last 10 years". It instantly devalued the entire program for me.

anacrolix2• 3 years ago

I have news for you... He's not wrong. The porcelain is different, but the same methodologies and processes are in place. The biggest change recently is distributed (mostly ignored) version control, that's 20 years old, and continuous integration/development (probably also around 20 years old, but only catching on in the last 10-15 years).

Computer science has changed more, there are lots of developments in the last 5-10 years.

cqqxo4zV46cp3• 3 years ago

Hah. This is classic knowitall CS/SE student hubris. They were almost certainly right.

corethree• 3 years ago

I know where you come from and I know where the people who are responding to you come from too.

Software has changed in the last 10 years, but a lot of it has changed superficially. A green software engineer most likely won't be able to tell the difference between a superficial change and a fundamental change.

It has a lot to do with the topic of this thread. "Quality Software" It's a loaded term. There's no formal definition, everyone has their own opinion on it and even then these people with "opinions" can't even directly pinpoint what it is. So the whole industry just builds abstraction after abstraction without knowing whether the current abstraction is actually close to "quality" then the previous abstraction. It all starts out with someone feeling annoyed, then they decide to make a new thingie or library and then they find out that this new thing has new annoyances and the whole thing moves in a great flat circle.

That's the story of the entire industry just endless horizontal progress without ever knowing if we're getting better. A lot of the times we've gotten worse.

That being said there have been fundamental changes. Machine learning. This change is fundamental. But most people aren't referring to that here.

davidgay• 3 years ago

"Software hasn't changed in the last 10 years". It instantly devalued the entire program for me.

As opposed to maths, physics, philosophy, civil engineering, classical studies which have gone through complete revolutions in their topics, problems and study methods in the last 10 years?

maxFlow• 3 years ago

Thanks for sharing, I'm reviewing the curriculum for the program [0] and it looks great.

Do you know if any of these courses has been opened sourced? It would be great to have access to part of the material.

[0] https://www.ece.cmu.edu/academics/ms-se/index.html.

zogrodea2• 3 years ago

Not the person you replied to, but I am aware that a functional programming course by CMU has had lectures (not exercises or content) freely available as YouTube recordings. https://brandonspark.github.io/150/

foobiekr2• 3 years ago

For years a friend tried to get his department (where he worked as a lecturer) to add a software engineering course where the basic syllabus was to (1) receive a TOI from the last semester and the code base, (2) implement some new features in the code base, (3) deploy and operate the code for awhile, and (4) produce a TOI for the next semester.

The code base was for a simple service that basically provided virus/malware scanning and included the malware scanner and signatures (this ensured there would never be an end to the work - there's always more signatures to add, more features, etc.)

I thought this was a fantastic idea and its a pity he never convinced them. That was more than fifteen years ago, and in his plan it would have just run forever.

maxwelljoslyn• 3 years ago

That sounds like an excellent way to do practice that directly mirrors real-world SWE, while still cutting it down to an appropriate size for a pedagogical environment. What a good idea.

mwcremer• 3 years ago

TOI?

rjzzleep• 3 years ago

The thing is for academics quality software sometimes isn't actually quality software.

My experience has been that people who's first jobs were in companies with quality software or who's job included reading through other people's quality software learn to write good software, the other ones learn whatever they saw in the environments they worked in.

steveBK1235• 3 years ago

I went to a STEM school and exactly 0 of the professors had been in industry ever or at least in the last 30 years. The only guy with some experience was an underpaid lecturer. He was also the only good lecturer.

A lot of professors just want to do research and mentor students onto the PHD track to self replicate. My mandated faculty advisor was basically like "go to the career center" when I asked about, you know, getting a job of some sort with my degree.

So yes, it is a real problem. CMU may stand out by actually having courses in the space, but it is not the norm by any means.

AlphaWeaver• 3 years ago

In my university, (US, state school,) we had a software engineering course exactly like this. It was great in theory, but in practice, the experience was rushed, the codebase was poor quality, (layers upon layers of nothing features with varying code quality,) and the background knowledge was completely ignored. The application we had to work on was a Tomcat Java Web application with an Angular frontend, when neither of those technologies were taught in any other classes (including electives.)

This approach to education can work, but I think simulating/mocking portions of this would have been more helpful (it could've been a teacher/TA managed codebase we started with rather than the monstrosity passed between generations of students who were inexperienced.)

podviaznikov• 3 years ago

100% this. My master program was for software engineering not CS. 15 years ago in Ukraine.

We have that. Maybe not enough. But it's definitely not a new thing.

bhk• 3 years ago

How many CS or CE grads from CMU are actually exposed to all of these topics?

Surely, "it is taught", but to whom and how widely?

tdeck• 3 years ago

It seems to vary a lot from school to school. At my university (2010-2014) we were writing unit tests from the first CS class. I can't say we got much instruction on how to structure more complex applications in order to make them testable, however. Things like dependency injection and non-trivial mocking have to be learned on the job, which is a real shame. Even within the industry skills and approaches to designing code for tests feel very heterogeneous.

prendino• 3 years ago

We were taught these things at the Bachelor's program in CS I went to in Sweden. At my first job I then slipped on a banana peel into a de facto tech lead position within a year, and I don't think it was due to my inherent greatness but rather that I was taught software engineering and the other colleagues at my level had not.

Ironically, the software engineering courses were the only ones I disliked while a student. An entire course in design patterns where strict adherence to UML was enforced felt a bit archaic. We had a course in software QA which mostly consisted of learning TDD and the standard tooling in the Java ecosystem, with some cursory walkthroughs of other types of QA like static analysis, fuzzing and performance testing. At the time it felt so boring, I liked to actually build stuff! A couple of years later I joined a team with very competent, CS-educated developers tasked to set up a workflow and all the tooling for testing software with high security requirements. They were dumbfounded when I knew what all the stuff they were tasked to do was!

rockwotj2• 3 years ago

Rose Hulman is an undergraduate teaching school that also has a distinction between a Software Engineering Degree and a Computer Science Degree. The Software Engineering degree takes a few less math courses and instead takes classes on QA/Testing, Project Management, and Formal Methods

skeeter20205• 3 years ago

SE is often taught out of the engineering school while CS the faculty of science, which in my experience makes a huge difference.

valand3• 3 years ago

It was a mandatory class too in the univ I attended.

What was not taught about were the techniques of QA's, the variety of tests and design pattern enforcements that would reduce QA cost by tenfolds.

nerdyadventurer• 3 years ago

Any plan to make it available to public, MOOC, lecture series etc?

tbm57• 3 years ago

To be realistic, it's important to not over-engineer QA measures with a big upfront investment. We shouldn't block progress of the project as a whole and neither will we get the necessary buy-in from all stakeholders with this approach.

I suspect that a lot of the bad code that is out there exists because teams are constantly in crunch time where it is important to get certain features out by a deadline.

From that perspective, this statement is kind of a contradiction. If every minute is vital to finishing a feature on time, then the act of writing tests will always block progress of the project

Basically, I just wish it were more innately profitable to write quality software

bradley13• 3 years ago

If every minute is vital to finishing a feature on time, then the act of writing tests will always block progress of the project

So you will deliver that feature, it will fail for your customers, and you will be in panic-mode trying to fix it on live systems. Seriously, that's going to happen.

Finishing a feature must always include at least some minimal time for testing. That testing needs to be done by someone not involved in the development. Developers sometimes misunderstand requirements (or think they "know better") and they will test what they implemented, not what the requirements actually say.

sanderjd11• 3 years ago

I have been, at various points in time, taught how to build quality software. I think the large majority of people I work with have been taught about this as well. So I'm not sure who the "you are never taught ..." is referring to here. Should it perhaps instead be "I was never taught ..."?

bluGill16• 3 years ago

Were you really taught that, or were you taught cargo-cult things that don't make for quality software. I've had some of each in my past.

This is a big area that I wish researchers would focus on. What really does make for long term high quality software. What are the actual trade offs. How do we mitigate them. What are the limits to TDD - is it good? Detroit or London mocks - when to use each? When is formal verification worth the time and effort? There is a lot places where we as an industry have ideas but are in heated debates without knowing how to find a truth.

ponector9• 3 years ago

My colleague, a senior full stack developer with CS masters degree has been asking: "why should I write tests if I can write new features?" And that is what managers often think. Because you can present new features to business but can't do the same with tests. They have no immediate value.

sanderjd11• 3 years ago

No, I have been, at various times, taught about a number of different techniques that are used in the endeavor to build quality software, along with lots of discussion about the tradeoffs between them and when they may be more or less appropriate.

It certainly is not something with a single easy answer, and I certainly agree that it remains a fruitful thing to continue researching, but that doesn't mean that there is nothing to be taught about it. There is lots to be taught about this, and lots that is taught about it, to lots of people.

bsuvc• 3 years ago

Huh. I was taught this in Software Engineering (2 semesters) in my CS degree, which was a required "capstone", so to speak, and focused on how to build real world software.

Some of it can't be taught though, any more than a craftsman or artist can teach a novice how to produce what they do with the same result.

The processes and steps can be taught, but only through experience can some things be internalized.

johnnyanmac5• 3 years ago

Neglecting QA is a shame because 90%+ of all students work in a company context after they finish their degrees. It will be necessary to deliver software without bugs in time.

once again bringing up the hot debate of "are colleges job prep or research institutions"? Many students these days will proceed to grab a job, but is that something a university should strive to be?

I wish at the bare minimum there were more proper apprenticeships for companies that want some specific type of software engineer instead of 3 month vibe checks as it is right now. Or bootcamps made more or less to study to the test instead of what actually brings value. But I guess no one is really chomping at the bit to change the status quo

bluGill16• 3 years ago

is that something a university should strive to be?

Universities taking public money (including students government grants/loans) should strive to make society better. Part of that is getting kids into good jobs that society needs done.

There are a few "retired" people taking classes that they are paying for on their own just for fun. If those people think they are getting value despite taking subjects society doesn't value I'm fine with that. The majority of students though are young people that society is trying to invest in to make a better future, and so the more universities prepare those kids to make a better future the better.

maximus-decimus• 3 years ago

People making roads and building powerplants also take government money. Do you expect them to prepare people for software dev jobs too?

Just because companies decided to start requiring degrees because they're too stuff to pay their own staff, IMO they shouldn't get to divert universities from their original mission, which is education and research. Afterall, those are also important to society, and if they don't do it, who will?

darth_aardvark• 3 years ago

Unrelated to the article, I could immediately identify the image used as definitely AI-generated. But I can't identify any reason why. It's a normal picture of a stone brick wall. Yet I'm 100% sure it's AI.

No shame to the author for their choice; replacing stock images with generated ones is a great use case. It's spooky to me that we've been so quickly trained to identify this subconsciously.

dvfjsdhgfv• 3 years ago

True. I think in this case it's because of the texture of the bricks. It looks like they were wrapped in cloth or something. This seems a common texture in many AI-generated images.

realprimoh• 3 years ago

AI-generated images have an unnatural "smoothness" to them whereas reality is more imperfect. At least that's my hypothesis.

hardwaregeek2• 3 years ago

I wrote a similar article: https://uptointerpretation.com/posts/art-school-for-programm.... I'd love to see schools that put programming first and foremost, that taught it with the respect that it deserves. There's this idea that programming is not worthy of a course of study, that programmers need to learn computer science and not programming. I disagree. It's a discipline unto itself and learning computer science to learn programming is really ineffective.

ck45• 3 years ago

This has been previously handled in the excellent book "The Problem with Software - Why Smart Engineers Write Bad Code" by Adam Barr (https://mitpress.mit.edu/9780262038515/the-problem-with-soft...)

withinboredom5• 3 years ago

I literally can't figure out how to buy the book from that site. I clicked ebook, then amazon, and the link is dead. You should probably just share a working link.

andrewl• 3 years ago

The Problem with Software - Why Smart Engineers Write Bad Code by Adam Barr

https://www.amazon.com/dp/026203851X

tanepiper• 3 years ago

Maybe a spicy take (FWIW I'm also self-taught)

Software engineering quality isn't something you teach, it's something you learn.

withinboredom5• 3 years ago

Usually, the hard way.

I've worked on so many poor-quality projects. I've had so many heated conversations trying to teach people that there are no such things as rules in SE and it's important to _know when to bend/break the rules_. Rigorously following rules doesn't mean high quality, automatically. Some things simply don't exist in the little world you're building.

Translations, for example, usually live outside of any layered cake you are making. If you write a string that will be shown to a user deep down in your code, it needs to be marked for translation and translated. If you try to layer it ... gods help anyone who comes along to change that switch statement.

There are lots of other cross-cutting concerns that people try to put into a box because of "rules" that don't belong in a box. That's usually where your quality starts to plummet.

bob1029• 3 years ago

I think developing high quality software is more art than engineering.

The most useful pieces of code typically have to deal with some of the most ridiculous constraints. Determining a way to integrate these such that the UX doesn't suffer is the real talent.

The only pedagogical model that makes sense for me anymore is the apprenticeship. You can definitely learn it on your own, but it's a hell of a lot faster to observe someone with 20k+ hours XP and skip some of the initial lava pits. Perhaps some of those lessons are important, but learning and knowledge are not linear things.

ponector9• 3 years ago

It is the same as you learn about backup and restore process. And then you learn hard way that you should have been testing your backups, if they can be used to restore data.

amelius• 3 years ago

What they also don't tell you is that quality software can turn into a nightmare if you let some manager add a bunch of random requirements to it, freely, after the software has already been built.

phkahler• 3 years ago

It doesn't help that early CS courses cover stupid examples like a recursive implementation of fibbonacci when the obvious solution is a loop. Tail call optimization is also another thing that should be reconsidered.

Plain imperative structured code is usually the cleanest and easiest to understand. Introducing abstraction too early I think confuses people and encourages complexity where it's not needed.

m0llusk• 3 years ago

This is a side issue, but it seems to me like this general problem is what is making strict typing so popular. If there were tests and documentation in place and parameters were being checked then strict typing wouldn't have much benefit because types are just one aspect of making sure the whole system is working as intended. Instead in many cases development is chaos with little if any documentation and testing and strict typing is great because it is the only tie holding the bundle of sticks together.

ChuckMcM• 3 years ago

From the article -- At some point, I realized that I wasn't using the right arguments either. Explaining that the software will be 'more stable' or 'make maintenance much easier' is not palpable for someone who doesn't work in the codebase themselves. We need to speak about money. As developers, we need to speak about the cost of not doing QA. This is the language of business and managers in general.

The more general way of saying this is, "If you need someone's approval to do something, explain it as a good idea for them." Took me a bit to learn this in engineering, you would think "but it will be correct!" would be an unassailable argument but as the author notes, the person who has to sign off on it may not care if it is correct, even if you passionately do care. This works for everything at the job, not just fixing software correctly.

sublinear• 3 years ago

I've found the only true way out of this hellish situation is to work at a place mature enough for the leadership to already understand all this.

If you have to explain why quality matters, they're at least as ignorant as you if not more. They deserve their fate. The upshot is you'll probably also get paid way better and more quickly develop a better sense of how business is supposed to be done at a mature company.

Of course you also have to deliver on your promises that all the time you're spending will improve things and isn't just some amateur quixotic itch.

t435622• 3 years ago

The reason I find it easier to work with people who have a degree in Computer Science is that I don't have to convince them of the need for good algorithms and not to try to implement parsers or cryptography by hand.

When it comes to software engineering I feel there is no qualification where you can feel that the gross naivety about quality and working in teams (and with other teams) has been similarly ironed out.

Instead you get people hardened in sin from their repeated experience of writing one horrible bit of untested code quickly and then leaving before the shit truly hit the fan :-)

One's management is very impressed with those kind of people and very contemptuous of those left behind who have to painstakingly clean up the mess.

ch4s33• 3 years ago

Hum, interesting perspective. I did 95% of a masters in CS before leaving to do a startup and while I can see the value of parser generators, there are a LOT of times when it is appropriate, useful, and more performant to write your own simple parser. It's often in my opinion the right thing to to first for simple cases. Clearly you should test it and consider functional requirements, but a stringy protocol with clear delimiters is usually dead simple to parse and depending on your use case you can focus on a subset. If you're writing a language... my advice might be different.

chiefalchemist2• 3 years ago

My impression is the gist is: When you think like an engineer, your focus is on problem solving, and using the appropriate tool(s) to do that. On the other hand, typical developers instinct is to code, code, code; at least based on my experience.

valenterry3• 3 years ago

I've never had it once in my career where using a parser generator wasn't better. Given that it's an in-language parser generator and not some meta-language monstrum like ANTLR.

Maybe when writing your own programming language, own complicated data format or low communication protocoll while requiring extreme performance. But that seems to be super rare, at least in my area of profession.

t435622• 3 years ago

I've worked with people who thought parsers were straighforward and trying to fix the bugs in their code was fraught with impossibility - there can sometimes be millions of ways for parsers to accept invalid input or not accept valid input.

In one case I gave up on fixing the code where every change introduced new possible bugs and used a parser generator. We never had another bug in that part of the code but my wholesale change caused intense friction.

I feel that a course in parsers would have helped that person to understand this wasn't an appropriate situation.

In fact I think it's a good idea to have BNF "that works" before you hand code anything just to confirm that you understand your own language design.

hyperthesis• 3 years ago

Pushback on parsers. It's very difficult to provide useful diagnostic error messages with yacc/bison. So most languages end up with a hand-written recursive descent parser.

The only exception I personally know of is jq (uses bison). So it's difficult to produce helpful syntax error messages in the implementation of jq.

quickthrower22• 3 years ago

It is a culture thing. Try to avoid cowboy shops. The thing is the general standard seems higher than 20 years ago. Source control, unit testing and CI/CD are not controversial any more for example.

pkkm2• 3 years ago

Cryptography yes, but are you sure about parsers? As far as I can tell, there's some kind of U-curve there. Beginners code them by hand, intermediate-level programmers and intermediate-scope projects use parser generators, and people maintaining the most sophisticated parsers prefer to code them by hand too. For example, GCC used to have a bison parser, but they switched to a hand-coded recursive descent one because that let them produce more helpful error messages. Clang uses recursive descent too.

mdaniel5• 3 years ago

I offer, again, my JetBrains GrammarKit counterpoint from the last time that assertion came up <https://news.ycombinator.com/item?id=38192427>

>>

I consider the JetBrains parsing system to be world class and they seem to hand-write very few (instead building on this system: https://github.com/JetBrains/Grammar-Kit#readme )

- https://github.com/JetBrains/intellij-community/blob/idea/23... (the parser I'll concede, as they do seem to be hand-rolling that part)

- https://github.com/JetBrains/intellij-community/blob/idea/23... (same for its parser)

- https://github.com/JetBrains/intellij-community/blob/idea/23... and https://github.com/JetBrains/intellij-community/blob/idea/23...

- https://github.com/JetBrains/intellij-plugins/blob/idea/233.... and https://github.com/JetBrains/intellij-plugins/blob/idea/233....

CipherThrowaway• 3 years ago

The reason I find it easier to work with people who have a degree in Computer Science is that I don't have to convince them of the need for good algorithms and not to try to implement parsers or cryptography by hand.

Cryptography and parsers simply do not belong in the same sentence. There is never a time when it is a appropriate to write your own cryptography. OTOH, most large compiler and interpreter projects have handwritten parsers, and many of them have handwritten lexers too.

Writing a parser can be simple enough to fit into a take-home assignment, and hand-written parser code ends up looking pretty similar to an LL grammar anyway. Parsing is also the easiest part of writing compiler or language tooling, so if a hand-written parser is too high a bar for the team then the entire project might questionable.

I'm not saying never use a parser generator, but I would personally prefer to work on a project with a well tested hand-written parser than a project using a parser generator. Especially if it complicates the build process with extra tooling, or is something really dated like Bison or ANTLR.

kaba06• 3 years ago

What’s dated about ANTLR?

codenesium• 3 years ago

I ask how do you build quality software in interviews. A lot of people are caught completely flat footed by it. They have no idea.

ponector9• 3 years ago

What if they ask you: and you? Are you actually building high quality software?

What are your quality gates? How many open bugs are there on production? How often you close bugs with resolution: won't fix(because of budget issues)? How often do you have production incidents? What is your testing strategy? How are you testing your requirements before assigning them to the developer for implementation? Do you hire external professionals for security testing?

Communitivity• 3 years ago

There is a corollary. You need to learn how to build quality software. You also need to know what level of quality vs completeness tradeoff you need to make.

Imagine I have a deadline of Jan 15th to demo features x, y, and z to senior leadership (who then will make funding decisions that could impact my project), and I get to Jan 15th with x, y, and not z - or worse, none of them working fully. But the code quality is high, there are no TODOs hanging around, no extra development focused logging, no duplication of code, no leaky abstractions.

That is a 100% fail in leadership's eyes, unless you have a really good story on why z didn't get shipped (and code quality is NOT that).

All those things I listed will have to be addressed at some point, and the longer they go without being addressed the harder they will be to address. But if you need to do that to meet deadlines, then you do it.

Of course, if you are in a place where leadership allows you to work from a backlog and demonstrations and features to demo are scheduled not arbitrarily based on leadership's schedule/interest, but on the features that are newly shipped since the last demo, then you are in luck.

At the end of the day the important thing to remember is that you are not being paid to build software. You are being paid to provide a solution to your customer's problem. Other than CTOs and some forward thinking leaders, they don't care about the software. They care about whether the problem is solved, did it cost me more or less than expected in labor and materiel, and is it compliant with necessary laws/regulations.

beebmam• 3 years ago

I think that "quality software" is not well defined, so until it's well defined, we're not talking about the same thing.

idlephysicist2• 3 years ago

Exactly, there's no point in a startup having _perfect_ code, great test coverage, and an excellent CI/CD pipeline if they've not gotten to market before the cash runs out.

Though I think that if leadership are not consulting the Engineers on timelines, instead just dictating the timelines to them, then there is a massive problem afoot.

adriangrigore• 3 years ago

Why get out of the hamster wheel?

eternityforest• 3 years ago

I completely understand university teaching algorithms over real dev stuff.

Algorithms are hard, and take serious work, they're basically math. University seems like the idea place to learn A* search or quicksort, I can't imagine figuring that out without a lot of effort and time spent just focusing on that.

What I don't understand is why programmers themselves focus on algorithms over stuff like "Vague heuristics to guess if this JS framework you tear your hair out in a month".

That kind of this isn't really hard and doesn't require any heavy discipline and focus, it's just something you pick up when you've tried a few different systems and notice stuff like "Oh look, that linter caught this bug for me" and "Hey look, there's not much autoformat support for Mako, maybe I'll just use Jinja2 like everyone else".

Instead they tell us to do stuff like code katas and endlessly polish our coding and algorithm skills rather than practice working on big stuff and managing complexity.

Maybe it's because algorithms are the hard part, and being really good at that is needed to work on the real cutting edge stuff.

ivix• 3 years ago

I suspect as with most things in education, they focus on small self contained problems because it's easier to teach and easier to grade. These toy problems end up being all students know and therefore are what employers select on.

satisfice2• 3 years ago

Almost no university teaches software testing in a competent way. For some years, Cem Kaner and I (he, a professor at Florida Tech; me, a proud high school dropout) ran a small annual conference called the Workshop on Teaching Software Testing. We specifically invited Computer Science professors to come and grapple with the problem. Thus, I personally witnessed and engaged with the sorry state of practice with teaching people about testing.

Cem developed a great testing class for Florida Tech, later commercialized under the name BBST, that drew in contributions from prominent industry voices. Otherwise, I don't know of any other University that has put the effort in. An exception may be what Tanja Vos is doing at Open University.

The problem I have with CMU is that they are the source of the Capability Maturity Model, which is simply incompetent work, justified by no science whatsoever, which tries to apply Taylorist management theory to software engineering. It never caught on despite all the marketing they've done-- it's only popular in military circles. I shudder to think what they are telling students about testing.

You can disagree with me, of course-- but in doing so you have to admit there is at least a schism in the industry among various schools of thought about what testing is and should be. My part of the field considers testing to belong to social science, not computer science.

austin-cheney• 3 years ago

Like with all things quality is proven through practice and measures, which means quality can only be guessed at before a product is built.

Like in all things here is how you do it:

1. Build a first version that works and accomplishes all initial business requirements.

2. Build test automation or inspection criteria.

3. Reflect upon what you built and watch how it’s actually used in production.

4. Measure it. Measure performance. Count user steps. Count development steps. Determine real cost of ownership.

5. Take stuff out without breaking business requirements.

6. Repeat steps 2-6.

That is how you build quality whether it’s software, manufacturing, labor, management, whatever.

In my own software I devised a new model to solve for this that I call Single Source of Truth. It’s like DRY but hyper aggressive and based upon empathy instead of micro-artifacts.

rcbdev• 3 years ago

In the Austria these topics are usually researched at universities of applied sciences (Fachhochschule) rather than universities (Universität). Some technical universities here attempt to combine these disciplines, to moderate success.

For example, the largest technical UAS in Austria offers Computer Science as an undergrad degree but only does research on Software Engineering at a graduate level.

jbs769• 3 years ago

As someone who never studied computer science formally, I find this perspective fascinating - for me, programming is the tool to solve the problem. It often feels like folks study the tool. And the problem is important too.

dudul• 3 years ago

What uni Teaches You blablabla

So I'm gonna need to see dome data here. How many universities has this guy surveyed to make a claim like that?

My university did teach me design patterns, architecture and all. My professors did give me projects where requirements would change mid way and the software had to adapt without being completely rewritten. They did teache to write unit tests, to use a build tool to generate docs, reports, etc.

Was I battle tested for my first job? No, but it really wasn't as bad as what's described here.

mushufasa• 3 years ago

I've found people with a computational physics background have a much better approach to QA. In that field "sense checking" calculation output is a part of the core methodology.

makach2• 3 years ago

What the hell? Yes it is taught. How it is interpreted afterwards is very dependant on the team or community you surround yourself with. You get all the tools from schooling, how we use them is entirely up to us.

makach2• 3 years ago

Obviously the entire article is an opinion piece and flame bait and I fell for it.

sinoue• 3 years ago

The competitive CS schools where many of the students needed good sample code projects have an innate sense of building quality software. Usually because they know it'll be showcased, but also because they haven't gotten lazy with shortcuts or been overly managed to spend time developing instead of fixing. I thought it was funny the article referenced the famous umbrella monster. Here is the longer clip: https://coub.com/view/284lib

readyplayernull• 3 years ago

I work in QA automation and also develop my own projects. There is more to quality assurance than QA. When you spend days creating something and then throw it away to make it better, that IS quality assurance! That deleted code, that deleted architecture, that deleted design is the cost of quality. A totally unappreciated aspect of QA.

RevEng• 3 years ago

I've been a software developer for 15 years. I started at a start-up company with many others who were fairly junior. We knew we had to take QA seriously, but didn't know how to do it. It took many years to come up with standards, tooling, and so on, but it was totally worth it. Everything in this article rings true from my experience.

efields• 3 years ago

Basic project management triangle stuff. We’ve been told by the biggest names in tech that first you move fast and money is practically bottomless. You get fast and expensive but quality suffers. The end.

benreesman2• 3 years ago

You learn this at quality shops. 10-15 years ago roughly FAANG.

Today? TailScale and stuff like that.

You can just noT have a bunch of pointless micro services and docker in your runc and layer on layer of json de/re, and unit test to get coverage but ignore quickcheck and hypothesis and fuzzing.

You can use stacked diffs and run oncall loops out of the team authoring the code and all of it.

You can minimize dynamic linking and all the other forms of unforced dependency error.

You can understand and play towards the runtimes of all the managed languages in your stack. You can insist that a halfway decent verbal grammar for the languages is “readability”.

It gets shouted down over and over but it’s public knowledge how to ship quality software.

ttymck• 3 years ago

I think the prevailing problem is that quality software often doesn't outperform shit software in terms of revenue, right?

benreesman2• 3 years ago

CloudFlare and TailScale and that crowd don’t seem to be chancing it?

foobiekr2• 3 years ago

It's not that, it's that quality doesn't matter and mostly neither does technology.

If you are doing network switches or GPUs or server CPUs or whatever, yeah, technology matters. If you are building pretty much any SaaS, MCCA, etc. the tech is literally irrelevant and mostly the more "new" tech you use the worse off you are.

Quality also only matters in some contexts and those are even rarer than the above.

Timing and solving a problem is all that matters in terms of revenue. As long as you aren't too bad, it's fine. The only real five-nines requirement for SaaS is that 99.999% of them would be fine with one or two nines.

CyberDildonics2• 3 years ago

roughly FAANG

What does roughly FAANG mean?

Today? TailScale and stuff like that.

The VPN?

You might have some ideas about quality software, but this comment is incomprehensible.

anacrolix2• 3 years ago

Someone's butthurt about never working for a company with a recognisable name?

phendrenad22• 3 years ago

OP is saying that 10-15 years ago FAANG companies (and a few more) were the only ones writing quality software. Now, FAANG doesn't care anymore but there are new unicorns in the making that do care - TailScale being one of them (debatable)

pkkm2• 3 years ago

Any advice on how to find these quality shops and this quality knowledge? Any particular things to look for, or particular books/courses you recommend?

jongjong• 3 years ago

Something which should be taught is the importance of coming up with the simplest abstractions possible and sticking to the business domain. IMO, if you can't explain an abstraction to a non-technical person, then you shouldn't invent it.

whalesalad• 3 years ago

I don’t think it can be taught, at least not in a classroom. Apprenticeship and learning from wise mentors in real world environments is the best way to learn.

mdaniel5• 3 years ago

In my opinion, there's a tiny bit of nuance: it can for sure be taught, but to be internalized requires experience, likely being on the wrong end of something sharp (pager, company going out of business, or you yourself experiencing a bug that otherwise only a customer would see)

phendrenad22• 3 years ago

Excuse me, I was taught how to build quality software, but I can't do it because the MBAs who run the company haven't been taught how to build quality software, and the board of directors who hired upper management were not taught how to write quality software. Most places I have worked the opinions of software developers were thoroughly ignored at best.

gjvc• 3 years ago

you can't teach software engineering before you have taught people programming. this suggests that programming should be a first requirement for a BS degree and "software engineering" should be MS and onwards.

pushedx• 3 years ago

I went to a University with required Co-Op experience in industry. There was a great feedback cycle of students coming back to classes and writing tests and design documents for assignments that didn't even require them. Reading articles like this make me really grateful for that.

sharts• 3 years ago

The easiest way to tell if you're about to join a bad team or company is to look at their QA process.

If it's slim to non-existent, you'll be working more hours with more on-call "incidents" and no life.

cognomano• 3 years ago

We need to speak about money. As developers, we need to speak about the cost of not doing QA.

Yep!!

dclowd9901• 3 years ago

I didn’t learn how to build quality _anything_ until I dove into woodworking. Order, organization, process, care, details. You can _work_ with mistakes, but you can’t undo them.

I highly recommend anyone wanting to learn how to do something with care, try to pick up a skill that requires you to do it right the first time.

vaxman• 3 years ago

QA/Integrations/SQM and how to avoid rejection was one of the skills taught hands-on AFTER university by the great masters who were unceremoniously forced into other occupations like: auto leasing, banner print shop, etc. when they were forced to crack open their 401Ks to make their house and family support payments in spite of the dot com bubble crashing the NASDAQ and related capX for six miserable years.

For example, at 16, my QA manager (whose husband had a Nobel Prize) and my tech lead (who wrote malloc/free for Unix and was learning VMS from me as fast as I could learn C from him) were totally opposite forces, separated by HR and different directors. QA didn’t code anything but batch jobs/shell scripts and contracted back to Engineering for any apps they wanted (or simply required them to be in the release package). If you wanted to keep your job, you didn’t submit programs with performance/functional deficiencies and definitely not logic issues as compared to the SPEC that all parties signed at the outset of the project (put that in yoh Kan Ban, Man rofl)

Ok another arrogant post from older former child prodigy! Happy Holidays!

gorgoiler• 3 years ago

This is a bit more abstract than the article but in my experience the best software comes from people with taste, rigour, reasoning, a good vocabulary, strong principles, and the ability to write clear and concise English (or whatever human language is used in their team.)

When you truly understand the software you are writing then, and only then, can you communicate it logically in code for the computer to execute and, much more importantly, code for a person to read. Well written and well understood code means it’s very obvious what you are doing. Later, when the code has a bug or needs to be rewritten, then it will at least be clear what you were trying to do so that it can be fixed or extended in some way.

So then the question is how do we train people to have these skills? In school, science experiments are a good way to teach logical reasoning and communication — here is what I thought would happen, here is what I did, here is what happened, and here is what it means. Math teaches you how to reason abstractly and, again, prove your point with logic. It’s a slightly different beast in that it’s harder for a math experiment to go wrong. It’s also harder to come up with and overcome novel scenarios in the lab with math, so in all it complements science well. And of course reading and writing English build your ability to express your thoughts with words and sentences. Many other high school subjects combine these in various measures — history for example is data gathering, fuzzy logical deduction, and reasoning in written language.

The bottom line is that quality software starts by working with well educated people and conversely all the most abhorrent heaps of over coupled illegible nonsense I’ve seen has come from people who, to be blunt, just ain’t that smart or well rounded, intellectually.

It’s a principle I carry over to hiring: smart and well educated wins out over pure-smarts.

ChrisMarshallNY3• 3 years ago

I really liked the umbrella demo :)

Here's my take on testing[0].

I write Quality software. It's a matter of personal satisfaction. Many folks can't really tell the difference between relatively good software, and very high Quality software, but the cost difference can be quite large. We don't get rich, writing top-shelf software.

The issue is, in my opinion, that we don't even produce much "relatively good" software, these days.

I won't get into it. It doesn't win me any friends. I was fortunate, to find a company that shared my values. Many of their processes would absolutely horrify a majority of folks on this site, and it was, quite frankly, often maddening.

But you can't argue with results. They produced extremely expensive kit, for over 100 years, and they are regularly used on the ISS.

[0] https://littlegreenviper.com/miscellany/testing-harness-vs-u...

tmshu1• 3 years ago

Thank you for sharing. I just read through several of your blog posts and especially resonate with your “evolutionary design”. The idea of integration tests/test harness first over unit tests makes a lot of sense to me too. As a one person team myself, the “art” of creating quality software products, at speed, is revealing itself and is quite fascinating.

It’s not everyday that devs like me get to learn from someone with as much experience as you have. Thanks again for sharing your knowledge!

ChrisMarshallNY3• 3 years ago

Aw, shucks, thanks.

But I am quite aware that I have a ton more to learn, and places like this, are a good place to do that.

I should get around to doing some more writing, though. I've been very involved in a project, for the last couple of years, and haven't taken the time to write.

Now that the project is approaching release, I may be able to free up some time.

ChrisMarshallNY3• 3 years ago

BTW: This seems to be the origin of that umbrella video: https://www.facebook.com/100001967368624/posts/2559941464081...

xav0989• 3 years ago

My university offered both a CS degree and a Software Engineering degree. While both open the door to tech jobs once in industry, the contents varied in the upper years. The software engineering one started with common engineering courses, then common CS classes, and finished with project lifecycle classes such as QA & requirements gathering. In contrast, the CS program started with a mix of core engineering and science classes, progressed to the common CS classes, then went further into advanced CS and algorithms classes.

askonomm• 3 years ago

The problem is that while experienced software engineers probably do know how, and would even like to, management never approves enough time for doing so, and so here we are.

neocodesoftware• 3 years ago

Sources?

References?

It sounds great - i am sure this has been studied. Anyone have any studies that have been done?

valand3• 3 years ago

A software business is made when someone needs an automated computation and buys a program from someone else.

The buyer has a specific requirement, which is distilled into a specification. The specification is implemented. Implementation that doesn't match with the specification is a bug.

Now in order to verify the implementation's correctness relative to the specification there must be a QA.

The idea that people forgets to add QA in the software development process is wild because it means people are forgetting how to conduct business.

oglop• 3 years ago

My favorite lines in here are people saying “we teach this, just get a masters”

Whoooosh

dep_b• 3 years ago

I did a bachelor in Software Engineering which is a bit more practical than Computer Science and while it really had quite a bunch of interesting practical subjects like communication, agile, waterfall etcetera there was absolutely nothing about QA.

SenAnder• 3 years ago

At some point, I realized that I wasn't using the right arguments either. Explaining that the software will be 'more stable' or 'make maintenance much easier' is not palpable for someone who doesn't work in the codebase themselves. We need to speak about money. As developers, we need to speak about the cost of not doing QA. This is the language of business and managers in general.

How cartoonishly incompetent have we allowed managers to get that they can't connect stability to money on their own? If the stability concerned a bridge, would engineers also be expected to translate a potential collapse into monetary terms to get their manager to approve a higher grade of steel cable?

dpweb• 3 years ago

Testing and QA is great and all however there is something to be gained from a CompSci education besides making more money for the company. It can be enjoyed just for the sake of it.

Binary arithmetic doesn't make dollars for your bosses, but its fun. That doesn't make it less significant.

rurban• 3 years ago

If you study coding at your university, you will be a stupid code monkey. However on proper universities you'll study much less coding, but Software Engineering or Computer Sciences, which teaches engineering and science. Not just QA, also mathematical foundations, writing proofs, writing science papers, writing compilers, doing AI, understanding HW, doing proper project management, with plans and tests.

Pity that he didn't go to a proper uni. Writing quality software is a solved problem, just not applied in most companies.

cloths• 3 years ago

From one point I think it's fair for Uni emphasise on CS over SE because there's "truth" inherent in CS, but for SE, which deals with building software in real life, is more complicated because real-life is complicated.

Even for the QA advice in the post "writing tests as you write the software" one can argue it's infeasible/inappropriate for my type of project and/or with the people who work on this project.

So my two cents on this is let Uni teach the students *to be aware of* all the SE principles and best practices this industry now have, like the tools in your toolbox, also let us know that whether to use them in real-life projects, need assessment of situation, cost/value balance, people etc, afterall not everybody works on dream projects.

Fire-Dragon-DoL• 3 years ago

I think there is a spectrum, it depends on the software it needs to be delivered and the need for business. A clear example are single player videogames, where the cost of automated testing is so high and the software has so little value after the sale that manual testing is a better option

You are never taught how to build quality software

Related Stories