Statistical process control after W. Edwards Deming

https://www.2uo.de/deming/

by tosh • 4 years ago

124 33 4 years ago

This is a cool write up, I think since his death people did not carry the legacy, and when I look at ai and machine learning it feels like the opposite of SPC.

There's a podcast I listen to by the Deming institute, the presenters talk about his legacy and work and rarely about new applications or techniques. Feels like the world of data driven management is heading in the wrong direction.

nonrandomstring2• 4 years ago

Definitely! Shewhart and Deming are massively under-rated and I agree that today's measurement-obsessive culture is heading "Into the Crisis" not out of it.

Every once in a while I read the 14 points , almost like a religious text to ask myself if I really do any of this, or have slipped "into sin". :)

djur• 4 years ago

I've always been interested in Deming and his ideas (my dad was kind of obsessed with him when I was younger) and every time I want to find some good material on him to get other people interested I have trouble. A lot of the websites, books, videos, etc. out there are clearly sales vehicles for individual consultants, and when you get past that a lot of the remaining material has a kooky, almost cultlike quality. And almost invariably it feels very old, not in the sense of time-worn wisdom but in the sense of something that has been left behind. The vibe I get is less "let's pass this wisdom on to a new generation" and more "we were right the whole time, you should have listened to us".

akolbe2• 4 years ago

Certainly there was an entire cottage industry of well-paid consultants popularising these ideas around the 80s and 90s and yes, there was something distinctly cult-like about the scene and the Six Sigma Green Belt and Black Belt trainings that grew out of them.

That's not the fault of the ideas though, many of which are just common sense.

I was involved in the production of interactive video training courses on SPC and related topics in the late 1980s and early 1990s ... the first ones still used 12" laserdiscs; the DVD and the internet hadn't been invented yet. :))

cardosof• 4 years ago

I've used Deming's ideas to optimize ads and it did work way better than bayesian ab testing or ML-based approaches. However, management did not like that I was using something so old, so I had to rebrand it, but it was Deming still.

nerdponx• 4 years ago

I find it extremely weird that "data science" as practiced in industry hasn't embraced statistical process control and operations research.

tomrod• 4 years ago

Some areas have. The whole DS field is a bit of a minefield, mostly due to a "newer is better" view of the world stemming heavily reliance on bootcamps/eschewing fundamentals.

May cooler heads prevail.

nonrandomstring2• 4 years ago

ISO 9001 and most of ISO 27001 are Deming through and through. If you tell some people you are going to use the principles of some old guy from the 1980s they laugh and dismiss you. Frame the same thing using some ISO numbers and throw in words like "compliance" and "best practice", and they go moist with glee. The shallow anti-intellectual fakery of some "management" sickens me.

dataflow• 4 years ago

ISO 9001 and most of ISO 27001 are Deming through and through.

Huh, why am I reading the opposite here https://www.oxebridge.com/emma/how-does-iso-90012015-stack-u...?

ISO 9001 has never much embraced Deming, despite them saying so. In fact, most of the requirements of ISO 9001 directly contradict Deming, as well as Juran, Crosby and other quality gurus.

haltingproblem• 4 years ago

Any chance of you doing a writeup on using Deming's idea to optimize ads? That would be a very interesting read as most ad optimization methods lack in intellectual rigor and controls.

edmundsauto• 4 years ago

I would love to know more about this - can you elaborate a little? Thank you!

przemo_li• 4 years ago

Can you give an example? E.g. signal that you chose to ignore?

zippyman552• 4 years ago

I used SPC charts at work for analyzing ton (1000+) of different computer generated jobs that had to run daily. The SW was always under modification and the charts helped to catch the errors and Pareto charts helped drive home people needed to fix their stuff. I have to agree w one of the posters: “ However, management did not like that I was using something so old” and this was always a struggle.

akolbe2• 4 years ago

Deming's quality management ideas (SPC specifically) were widely adopted in the US automotive industry by the 1980s and 1990s.

Later on this entire field came to be known as Six Sigma, but Shewhart and Deming are still the bedrock underpinning much of it.

deterministic2• 4 years ago

If you are interested in this you might also like the book “The Goal”. It is a business novel introductions TOC (the Theory of Constraints) which IMHO is the #1 most efficient way to run a company/team/process/…

kqr2• 4 years ago

The problem with the theory of constraints is that it, from my understanding, uses inner constraints to determine boundary ingress rates. This causes higher variation than necessary. When a bottleneck is introduced, you will have a lot of crap backed up in front of it because you ran upstream processes at high rates into the bottleneck – even though you knew the bottleneck was there. When the bottleneck is lifted, you instead have upstream processes starved waiting for the flow from the boundary to come.

A better alternative in the face of some variation is local signalling like kanban. This will run upstream processes at just the right rate all the way back to, and including, boundary ingresses.

steppi2• 4 years ago

This problem actually comes up in The Goal. After the characters lift the bottlenecks in their factory, things go smoothly for a while but then they start to see starvation in other areas like you mentioned. The solution they come up with involves local signaling and also data collection and analysis to understand the timeframes in play within the factory. The following excerpt is from Chapter 26.

Then I said, "Fine, but how do we time each release of material so it arrives at the bottleneck when it's needed?" Stacey said, "I'm not sure, but I see what you're worried about. We don't want the opposite problem of no work in front of the bottleneck." "Hell, we got at least a month before that happens, even if we released no more red tags from today on," said Bob. "But I know what you mean. If we idle the bottleneck, we lose throughput." "What we need," I said, "is some kind of signal to link the bottlenecks with the release-of-materials schedule." Then Ralph, to my surprise, spoke up and said "Excuse me, this is just a thought. But maybe we can predict when to release material by some kind of system based on the data we've kept on both the bottlenecks."

dsr32• 4 years ago

If I am interested in learning Deming ideas and theory, what is the textbook/book that I should read? Any recommendation?

wsh• 4 years ago

For Deming’s own ideas, such as the 14 Points, see his book Out of the Crisis (1986), reprinted by MIT Press.

For the practical application of control charts and other techniques (due originally to Shewhart and others), see the Statistical Quality Control Handbook (Western Electric, 1958) or, for a more up-to-date treatment, the NIST/SEMATCH e-Handbook of Statistical Methods:

https://www.itl.nist.gov/div898/handbook/index.htm

dsr32• 4 years ago

Thank you, I am more interested about the latter (implementation of his techniques at industrial setting). Will check that soon.

jimmygrapes2• 4 years ago

The Essential Deming[1] is probably the gold standard for now, though it can be a little dry. It's one of those "suggested readings" in basically any safety or process management higher education curricula.

Not really 100% about Deming but very relevant, I would recommend Alfie Kohn's "Punished by Rewards" [2] as a supplement to understand some of the implications of what happens when Deming's ideas are implemented without understanding the human condition.

Otherwise, Deming wrote and published plenty of his own work that is worth reading.

[1] https://www.amazon.com/Essential-Deming-Leadership-Principle...

[2] https://www.amazon.com/Punished-Rewards-Trouble-Incentive-Pr...

zippyman552• 4 years ago

I passed my copy on, but here is a nice 30 minute read where Tokyo waitresses were using the principles to sell sake.

https://www.amazon.com/Spc-Esquire-Club-Donald-Wheeler/dp/09...

throwaway892238• 4 years ago

I'm so happy when people bring up the "...wtf" side of Deming, in context of his more useful contributions. Hero worship is bullshit. A person who has really great ideas also has really crap ideas, because nobody is perfect. We should be more comfortable humanizing people, so we don't get trapped in the cargo cult or appeal to authority, and so we can accept more ideas from people who don't look like geniuses.

All that said: I find it so fascinating that his focus was on statistics and science, when so much of his advice is actually holistic observations from experience, and not all the result of scientific experiments or research. There is both a science and an art to making an organization (read: a group of people doing organized work) work well. You really need both, because people are, unfortunately, not science experiments, but weird blobs of meat and emotions interacting in an unfathomably complex system of systems of systems.

tgv• 4 years ago

That's probably the first text on business/management that I found interesting to read. In my (limited) experience, such texts are usually extremely dry, and lack any logic. Often they seem to recommend abstract rules without any reason. This text was more "reasonable", and the funnel experiment is very suggestive.

BTW, why woiuld the author wonder about the multiplication of loss and probability? Why would it be a convolution? I suppose the intention is to minimize expected loss.

sampo• 4 years ago

I would think you need a bit more machinery, like a real convolution, wouldn't you?

No. I think pointwise multiplication of functions is what you need here. Just as Deming did.

cdragonetti• 4 years ago

It's easy to think of Deming as "old fashioned" or irrelevant in the modern world (and yes, some people talk about him like a cult figure), but his ideas are fundamentally practical, enduring, and flexible. The Deming Institute over the last few years has shifted from "preserving Deming's teachings" to "teaching folks how to use Deming in their businesses/organizations now."

I recommend visiting www.deming.org, and also checking out DemingNEXT (https://deming.org/learn/demingnext/). It's a new, unique way to learn Deming online with interactive courses, case studies, "what would you do" scenarios, examples, articles, podcast episodes, and more. (You can get a free 14-day trial to check it out.)

I also recommend watching "why Deming, why now" video here: https://www.youtube.com/watch?v=7zCD2w3u9Ms&t=3s.

Full disclosure - I've worked for The Deming Institute for just over a year now and I've been blown away by how simple but powerful his ideas are.

Statistical process control after W. Edwards Deming

Related Stories