ismaelct

Very nice

I understand the sentiment but I think it's highly dependent on context. Where you work at, who you work with, org size, what the problem is, the cost and benefits of each abstraction, etc etc. I think it's our job as developers to put all those things on the scales when deciding what abstractions to use or not use. Whether a pattern is "the default" or familiar is certainly a big factor, but not the only one.

What do you think of Ruby's built-in function composition? https://ruby-doc.org/3.2.2/Proc.html#method-i-3E-3E

To be clear this is not a library. I'm just describing a pattern and a bare-bones implementation.

Well spotted. Fixed. Thank you!

Nice. Yes the pattern I describe in the article supports any callable too. I should point out that this is not a specific library, just a very bare-bones pattern.

If all those steps in your pipeline are defined in the same codebase

I find that rule a bit arbitrary. For example, I have data-import pipelines where some steps are unique to each task, but some others are shared across tasks. Why does it matter whether the steps are defined in the same codebase or not?

I just wanted to make the either/or pun.

Thanks Mitchell (big fan, btw!).

Indeed!https://github.com/hashicorp/vagrant/blob/main/lib/vagrant/a...

Yes, it's not a new pattern by any means, and there's many ways to "halt" the pipeline as you say. For example ActiveRecord stops callback chains if any callback throws ":halt".

Other examples are Redis.rb's pipelining API https://github.com/redis/redis-rb?tab=readme-ov-file#pipelin...

Or more generally any builder-style pattern that composes a set of operations for later execution, including again ActiveRecord's query chaining.

In my article I tried to show a specific implementation using the Railway pattern (where the result must only respond to "#continue?() => Boolean")

Odd to see ROP and monads as an either/or problem (see what I did there?). Most ROP implementations I've seen rely on the result monad.

I'm so going to steal this

Good observations, thank you. I take the point about properly delimiting the boundaries of the orchestration layer in a system.

I think you're right about how that line comes across, I'll try and improve it.

I tried to keep this as idiomatic to Ruby as possible. No meta-programming, no FP machinery like monads, etc. Little more to it than an array of objects and a reduce function. All plain Ruby in my book.

I wonder if just using "throw :halt" (like in AR callbacks, to halt callback chains) or an Enumerator with "raise StopIteration" (which would work) would throw people off a bit less, just on account of those things being more familiar.

Thanks. Where can I find your library? I'd love to compare notes!

I agree with the drawbacks listed here. I would add that "careful consideration" of any pattern we use is the job description.

I would further add that we should extend that thoughtfulness to the opinionated frameworks many of us rely on. They usually come with hundreds of complicated patterns baked in, and we "oversimplify our interfaces" to match the framework as a matter of course. All I'm saying is that committing to any mental model provided by a design pattern or framework has similar drawbacks. In this case, the mental model is "a big operation is a list of smaller operations in a chain". Use with caution.

No need to think!

Not sure I agree with fully. If anything, I've found that I need to think harder about what constitutes a step in a workflow, what are the names of each stage, what concept they encapsulate. I can't just chuck everything into a god object or a deeply nested hierarchy tree somewhere.

But again I agree. Using this pattern I've definitely over-complicated or gone down the wrong path at times. I would say though that I found it easier to roll back and change direction when compared deeply nested object graphs, for example.

Very good article, thank you.

I would add that the tagline says "when used thoughtlessly".

For example, I agree that using Result objects to reinvent exceptions is a bad idea. There's a reason they're called "exceptions". Error results should only be used for domain-level errors. ie. things that _your expect_ to go wrong in the domain.

Also note that in my article I use continue/halt, not Ok/Error. At the library level it's just a way to compose functions together with a mechanism to halt processing. Whether something halted execution because it was an error, or any other reason (ex. caching), is up to your app's semantics.

Amazing.

What do you think of the Rack interface?

Or ActiveRecord's query chaining?

ex. User.where(admin: true).joins(:account).order(id: :asc)

It's all versions of the same approach.

And yes all of the above _can_ complicate debugging, for sure. But so can most abstractions. Only use them if the specific problem they solve is bigger than the drawbacks.

I would have thought that "senior" means assessing things in context instead of falling back to truisms. "over engineering" only makes sense relative to a concrete problem you want to solve.

Good observation about stack traces and abstractions. Re. your question, the pattern itself is no different than, say, Rack middleware, so you'd see similar cost and benefits. In essence you're running one callable object after the other.

A pipeline is essentially this

steps = [ ->(r) { r }, ->(r) { r }, ->(r) { r }, ]

Wrap initial data in a common Value object

initial = Result.new(some_data)

Run the Result through the steps, in order

result = steps.reduce(initial) { |r, step| step.call(r) }

That's the pattern, really. A reduce operation.

Re. stack trace, it can add noise because you're iterating over steps instead of calling them procedurally one by one, and you may want to decorate steps (put steps inside steps) for encapsulation, caching, etc. but again no different than Rack.

Thanks for your feedback. Out of curiosity, when you say "plain Ruby code" what do you mean, exactly? Presumably you're still making use of _some_ patterns that you think are Ok.

Perhaps I phrased this badly. I don't think the entire article boils down to inheritance vs composition. But in discussing these patterns elsewhere, some of the pushback has been that many Ruby devs prefer to decompose problems via sub-classing instead of composition of command objects, so I tried to cater to that objection with that line.

Pipeline steps are command objects (a pretty standard OO pattern), so they have a single entry point / public method. But they can still fully leverage any other OO pattern in their implementation. The more complex ones I use may instantiate other objects, pass messages between them, etc.

But the single-method #call API is what makes composition easy. See Rack, or any number of middleware-style designs, for other common uses of this in Ruby.