The Zen of Parallel Programming

https://smolnero.com/posts/the-zen-of-parallel-programming

by edgar_ortega • 8 days ago

189 36 8 days ago

_def• 3 days ago

This resonates with me. On the one hand I feel like my team is spread too thin all the time, scope and complexity is just too broad - and at the same time I'm pretty sure nore headcount could only improve it to a certain degree.

I like the part about synchronization and honesty. And I'm certain every time I get annoyed that someone is not as open with me as I wish they were, there is a part in there that myself contributed to that.

mycall• 3 days ago

CS101 Day 1: Learn about Divide and Conquer in depth. Honesty forms the ability for controlling signal strength, avoiding impedance mismatches or logically, synchronization between the divided slices.

quorumsensor• 3 days ago

Another parallel from the Western tradition: ‘homologia’, translated maybe best as congruence or coherence. A Greek Stoic concept of the mind/body/entire organism in alignment, caused by reason and feeling being in harmony.

hinkley• 3 days ago

There's a woman I read about a while back who did here PhD thesis on the notion that David Hume's contribution to the Enlightenment was inspired by conversations with a Jesuit monk who had returned to France from a long engagement in a Buddhist area of India. DDG is telling me it might be Allison Gopnik but I've no recollection of the name, only the contents.

She proved that he made a trip to a town where the monk lived after his return, and the library contained a copy of his reports. She could not prove more than opportunity but she strongly suspects that Hume not only read the report but may have met with the monk as well.

patcon• 3 days ago

I found elsewhere referring to her as an "independent philosopher", but maybe Rachel Paine? https://archive.philosophersmag.com/was-hume-a-secret-buddhi...

assimpleaspossi• 3 days ago

I don't understand what you just wrote has to do with anything here.

datadrivenangel• 3 days ago

Fred Brooks and the mythical man month observed that communication between people expands faster than linear, and that adding more people to a project makes it later.

ctkhn• 3 days ago

I wonder how much a good process of meetings, tickets or whatever sort of documents etc for handling new ideas and features would speed this up or bring it back to linear. I've been in some orgs where good organization at the level above me can make the required communication between me and other dev teams lower and let us live in "good" silos

toast0• 3 days ago

What really seems to make a difference is how well the project and the organization are at being split into indepedent groups.

I've been on projects where everybody needs to know everything, and on projects where many groups can be insulated from others.

The more your group can work without needing a meeting with other groups, the more things can progress in parallel.

Some projects won't work well with that constraint though. And some organizations won't either.

reinitctxoffset3• 3 days ago

I think it's kind of fun now. Agents can interact with Linear. I'm still playing around with like does an issue achieve anything? What is the fastest way to get a bug from the observing session to the originating session?

But it's secondary.

The wall is build. If you can AI program, your problem is the build is too slow, too unreliable, not secure enough from a supply chain standpoint.

That is where one competent senior hacker tops out today. The agents are yielding to a CI that gets 35% per-vCPU occupancy.

The wall right now is skill or build, depending on your skill.

AdieuToLogic• 3 days ago

Fred Brooks and the mythical man month observed that communication between people expands faster than linear, and that adding more people to a project makes it later.

In my experience with more than a couple dozen software projects, the ideal team size to successfully deliver a non-trivial effort is between five to nine people.

  One subject matter expert visionary
  One technical architect visionary
  One or two technical masters
  One or two UI/UX masters
  One or two padawans capable of becoming a master

Staffing exceeding the above for an individual team is more a managerial genitalia contest than anything focused on organizational success.

locallost• 3 days ago

Amdahl's law says something similar, adding more processors to a problem eventually gives almost no speedup because parts of the problem cannot be parallelized and need to be done sequentially.

To the people wondering the meaning of the article, it's I think this.

CyberDildonics• 20 hours ago

amdahl's law is trivial tautological nonsense and has nothing to do with the mythical man month.

The mythical man month is about communication between people and the amount of work to be productive in a project. It's about people colliding with programming.

It has nothing to do with a vague statement that "eventually" you have diminishing returns on processors (of course there is some limit even though different tasks could be many orders of magnitude apart).

FullGarden_S• 3 days ago

Communication in this sense is pretty common in Mathematics and Computer Science and it becomes apparent when its related to actors and their interactions.

A good example is game theory and game semantics. The term "actor" in this context are soo abstract that we can pretty much attempt to implement them anywhere we see fit and that I think is the beauty.

torginus• 3 days ago

I cannot make sense of a single word of this essay. What sort of insight am I missing? There's not a peep here about write barriers, futexes, OS schedulers, critical paths and/or how they relate to Zen or whatever. This article can be easily recycled as the Zen of compiler design or Zen of audio engineering or whatever, without substantially having to chance much of the words, thats how generic it is.

I'm genuinely considering that this is just a foreword and I missed the clickthrough link to the meat of the article.

roadbuster• 3 days ago

There is no substance. It's navel-gazing dribble.

The "about" section of the website is equally vacuous:

    SmolNero is a philosophical startup rooted
    in first-principles thinking.

    We explore how humans relate to technology—
    how we speak to it, depend on it, thank it,
    and sometimes forget it’s there at all.
    
    We don’t claim expertise. We see ourselves 
    as translators—working at the crossroads of
    emotion and programming, curiosity and care

(The two M-dashes are also suspicious)

brobdingnagians• 3 days ago

There have always been snake oil salesmen, for awhile they had to put the effort into knocking on your door peddling MLMs, now they're making more impressive snake oil, but vacuous, with LLMs.

DavidVoid• 3 days ago

Seems to be run by a married couple of (maybe) burnt out account managers. That, in combination with some AI help, explains the empty "LinkedIn slop" writing style I think.

It's not often I comment on blog posts, but this is honestly one of the most meaningless blogs I've ever read (not counting those fully AI-generated SEO ones). I don't get why it ended up so high on Hacker News; it has an interesting title I guess, but pretty much no substance.

gwbas1c2• 2 days ago

What sort of insight am I missing?

The article makes a lot more sense if you've practiced Zen.

Think of it as someone's way of making sense of Zen teachings by related them to something they already understand.

bunderbunder2• 2 days ago

It is. The next post in the blog expands on the idea in concrete terms. The author doesn’t explicitly say so, but based on the shape of it I would assume all we have right now are the introduction and first full post of what is intended to be a multi-part series.

rramadass• 2 days ago

You are absolutely right.

It is just some daydreaming verbiage trying to connect totally disparate domains in a fanciful manner; nonsense and drivel.

As I continue reading An Introduction to Parallel Programming, I cannot help but notice a connection between communication among processors, communication among human beings, and communication within the individual self.

I say ... Whut?

SuperNinKenDo• 3 days ago

I appreciate somebody trying to connect disparate concepts to give both some kind of fresh perspective; perfectly happy to entertain analogies that break down if you look at them too hard... but like somebody else said, this reads like a draft or a foreword. Also, what about any of this is "zen"? This goes nowhere and has no connection with the concepts put forward in the title. Chucking "the zen of" in front of anything for no reason was tired 10 years ago, yet people keep doing it. We get it, you've heard of the book.

pjio• 3 days ago

Aside from the sentence "How many experiences continue to consume us because they were never allowed to finish burning?", I'm not sure what to make of this article. But the book it mentions, "Zen Mind, Beginner's Mind", seems worth taking a look at (an older one, not a promotion).

gwbas1c2• 2 days ago

Yes, the article really only makes sense if you're both practiced Zen and done some intense parallel programming.

"Zen Mind, Beginner's Mind" makes more sense in a group setting. I read extracts of it with a Zen priest in a meditation group.

baud9600• 3 days ago

It’s true that “zen of” is bolted onto many topics… it’s a tired cliche to suggest “simplicity” when things are complex. Does it help or is it faux spirituality?

I’ve enjoyed reading the comments here and I think there’s truth in how the technical problem is divided and teams are arranged. The idea of frequency of features (or builds) being a reflection of our division of the problem, is interesting. It made me think about our teams trying to ship releases and the problems arising, but zen and parallelism don’t give any hints. It’s just about effort to organise better, like it always was

bunderbunder2• 2 days ago

(Speaking as a lapsed participant in a zen community. Opinions are my own, so others may disagree.)

The challenge for these “Zen and my job” types of work is that much of Zen practice is (deliberately) a bit of an inkblot test. If there’s any real Big Truth to it, it’s that it’s all one big muddle that everyone has to muddle through for themself. Unlike in many Western religious traditions, there is no presumption of univocality - even within the canonical literature, every opinion is understood to be the author’s alone and represents their own personal attempts at muddling things through. It’s just that some people and works are regarded as being particularly worth a read.

But that’s never explicitly stated. Which can be particularly confusing to people who grew up in religious traditions that do presume univocality and a greater degree of spiritual authority. Hence these works by people who see one particular bit that clicks for them and conclude, “Ah, this is what Zen is really about, I must go tell others!” without realizing that perhaps they just stumbled across an inkblot that they found to be particularly evocative.

socketcluster• 3 days ago

I try to find an embarrassingly parallel solution to most problems I encounter. Not only because such solutions scale, but because they often produce a simpler, more robust architecture which helps you to avoid future issues. It's great for avoiding single points of failure and performance chokepoints. Also, there is usually little to no overhead for choosing a parallelizable solution (besides a little bit of additional up-front thinking.)

To people who say "You don't need scalability" I say "You also don't need unscalability..."

alexpotato• 2 days ago

Best quote over on distributed systems as applied to humans (from Dan Luu):

"Everything we've looked at so far is a technical problem. Compared to organizational problems, technical problems are straightforward. Distributed systems are considered hard because real systems might drop something like 0.1% of messages, corrupt an even smaller percentage of messages, and see latencies in the microsecond to millisecond range. When I talk to higher-ups and compare what they think they're saying to what my coworkers think they're saying, I find that the rate of lost messages is well over 50%, every message gets corrupted, and latency can be months or years"

- from https://danluu.com/sounds-easy/

kiaansaraiya• 2 days ago

Something that never gets old is Rob Pike's framing: concurrency is about structuring a program, while parallelism is about the execution. Plenty of concurrent programs never run in parallel and there are plenty of parallel programs that are poorly structured.

The Zen of Parallel Programming

Related Stories