ES7 Proposal: The Pipeline Operator

https://github.com/mindeavor/es-pipeline-operator

by clessg • 11 years ago

80 73 11 years ago

k__2• 11 years ago

I liked this feature in LiveScript rather much.

Especially that it worked in both directions.

    my-list = filter <| sort <| get-data

1971genocide• 11 years ago

Doesn't LiveScript already have it ?

chc• 11 years ago

As noted in the OP, yes, as do F#, Elixir, Elm and Julia.

dustingetz4• 11 years ago

Can we just put more focus on sweet.js so we can do this kind of in a library with standard build tooling? http://sweetjs.org/

There's a lot more operators than |> that you would want. Clojure commonly uses ->, ->>, as->, etc. It should be done in a library if it is done at all.

Also, going full-on functional in javascript is going to end badly, javascript simply wasn't designed for it. If you go down this path in real projects, sooner or later you inevitably end up at clojurescript or whatever other real FP language suits.

wycats8• 11 years ago

Can you elaborate on the rough seams between JavaScript and "full on functional"?

I'm not a believer in the FP vs. OO religion, and have historically been skeptical of the bind operator. That said, I think that both composition via objects and composition via functions have their place.

The pipeline operator attempts to make composition via functions more readable (left-to-right instead of right-to-left) and unless you think composition via functions is a bad idea in JS in all cases, I'm not sure what "full-on functional" has to do with it.

dustingetz4• 11 years ago

js objects are mutable; utliliy libraries like underscore, lodash, jquery, do not provide purity guarantees, they export impure functions that mutate objects (e.g. https://lodash.com/docs#merge). When we start leaning heavily on function composition to build our abstractions, we simply must have purity guarantees for everyday utilities or everything falls apart. So to go "full-on functional" in javascript means we need to fork and rewrite the entire ecosystem. And if you're going to do that, there isn't much value in keeping javascript.

gotchange3• 11 years ago

I think maybe he means that it is not idiomatic. It doesn't feel like JS with the proposed Ruby-like new bind syntax and now with this Unix-flavored pipeline operator, the language is taking on a new identity that some may not like.

tragic2• 11 years ago

Briefly, you can look at it through the prism of equality.

In javascript, 1 === 1, and "foo" === "foo". However, [] !== [] and {} !== {}. This is because in Javascript, like in Java, there is a distinction between value types (essentially, simple primitive values like numbers and strings) and reference types (collections - arrays and objects). When you assign an array to a variable, you are actually pointing the variable to a reference to the array. As a mutable data structure, its identity is more important than its value: it is expected that you shall be calling push, etc, and thus changing its value in place.

In Clojure (for example), (= 1 1) and (= "foo" "foo") and (= [] []) and (= {} {}). Because Clojure's data structures are immutable, the idea of checking for identity is almost meaningless (although you can, with the identical? function). In an FP style, with generic but immutable data structures, we are more interested in value equality, since we are not concerned so much with objects with a long lifecycle over which the values of their fields may change.

Javascript is in an odd position - it has enough features for both traditional OO and functional programming styles to be possible, but the two do not easily co-exist. Compromises are necessary. So, if our program is composed of mostly pure functions operating on generic data structures, we must do a lot of defensive copying, and use library functions for deep equality, because the data structures are not designed for this usage.

This is not to say it's infeasible to program in this way in Javascript: it is, and I try to. But a language designed from the ground up for functional programming will point you more enthusiastically in that direction than JS does.

On topic: like the syntax!

renekooi• 11 years ago

I like this a lot. It's similar to the proposed function bind syntax[1], but a bit more friendly to not-`this`-related use:

    function bindMap(fn) { return this.map(fn) }
    [ 1, 2, 3 ]::bindMap(n => n + 1)
    // standalone use: bindMap.call(arr, fn)

    function pipelineMap(fn) { return arr => arr.map(fn) }
    [ 1, 2, 3 ] |> pipelineMap(n => n + 1)
    // standalone use: pipelineMap(fn)(arr)

(Also nice that it works in a very similar way to decorators, although they won't be interchangeable in most cases, with decorators working on class constructors and property descriptors, instead of just any value.)

Presumably, at most one of these proposals will make it in :)

[1] https://github.com/zenparsing/es-function-bind

wycats8• 11 years ago

I agree that a "functional" operator that uses `this` as the first argument is weird.

The main goal of all of these proposals is "left to right" composition when working with functions, and functions take parameters.

RodgerTheGreat• 11 years ago

In concatenative languages, this style of composition is the norm, and it does indeed work well for many tasks:

http://concatenative.org/wiki/view/Pipeline%20style

lisivka2• 11 years ago

It is available in JS too:

    function double(str) {
      return str+', '+str;
    }
    function ucase(str) {
      return str.slice(0,1).toUpperCase()+ str.slice(1);
    }
    function exclamation(str) {
      return str+"!";
    }
    
    var _="hello";
    _ = double(_);
    _ = ucase(_);
    // _ = foo(_);
    _ = exclamation(_);
    
    console.log(_); // Hello, hello!
    
    function pipe() {
      var functions=[];
      for(var i=0; i<arguments.length; i++) {
        functions[i]=arguments[i];
      }
      return function pipe_call(arg) {
        for(var i=0; i<functions.length; i++) {
          arg=functions[i](arg);
        }
        return arg;
      }
    }
    
    var _="hello";
    _ =pipe(
      double,
      ucase,
      // foo,
      exclamation
    )(_);
    console.log(_); // Hello, hello!

donatj• 11 years ago

I feel like that just confuses things for no good reason. It's no fewer characters than parens but provides as I see it less clarity.

efdee2• 11 years ago

I think the clarity comes from the reading order. str |> method1 |> method2 |> method3 vs method3(method2(method1(str))).

mbreese2• 11 years ago

That's readable in the simple case, (arg1 is a string, returns a string). What about the more complex cases? Different method signatures, required extra arguments, returning objects or arrays? This will either a) break in those cases, or b) not be applicable. In which case, it's extra complexity for very little gain.

Seriously, this just seems like syntax sugar that just isn't needed. I get that you might not think it's elegant, but is anyone seriously troubled or confused by method3(method2(method1(str)))? How would the pipeline syntax help here?

whichdan• 11 years ago

Yep; same reason Lodash implements _.flow and _.compose.

It also encourages you to write methods that take a single argument and return a single value, or partially apply those that don't.

coldtea• 11 years ago

How it's "less clarity"?

What happens is much easier to read if you know what the operator means.

The only way the old way has more clarity is in the "we are already familiar with it" way. In any other way, this method trumps it.

bratsche• 11 years ago

It's much less clear than the F# and Elixir versions of |> are, so if you've used those languages this feels confusing looking. It would be nice to introduce |> into Javascript while keeping it just as understandable as it is in every other language that already has it.

pskocik2• 11 years ago

What's so painful about just doing `exclaim(capitalize(doubleSay("hello")));`? Why introduce needless syntactic sugar that'll only confuse people about how UNIX pipes work?

thescriptkiddie• 11 years ago

(because (there (is (such (a (thing (as (too (many parens)))))))))

pskocik2• 11 years ago

I bet Lispers would beg to differ. :D

Nested function do one thing on a chunk of data, then another, then another.

*nix pipelines apply all the filters at once and control the scheduling and lifetime of the filters.

Those are quite different concepts, and I think it's confusing to conflate the two. Plus every decent editor can handle paired parentheses.

efdee2• 11 years ago

I think you meant many_parens(too(as(thing(a(such(is(there(because)))))))).

radicalbyte3• 11 years ago

To understand the nesting you have to evaluate the functions in the opposite order than you're reading them:

exclaim capitalize doubleSay

Sure, some of us has spent 20 years doing this so it's easy, but you can't deny that this is far nicer to read:

doubleSay capitalize exclaim

I'm a big fan of Fluent Interfaces, so I use this construct fairly often:

"hello".doubleSay().capitalize().exclaim()

andrewpe• 11 years ago

This should be how it's done, simplistic, nothing new to learn, easy to read.

edgyswingset2• 11 years ago

Because that's a terrible code smell! The proper way to handle that case in a language which doesn't support pipelining would be to break each call into a variable and use those those variables. But even that can be tedious, hence the pipelining operators which have cropped up in some languages.

abrezas2• 11 years ago

it's very common in javascript to have chained methods, to be able to do things like

    _(array)
    .compact()
    .map(function(x) {
       return x * x;
    })
    .filter(function(x) {
        return x > 100;
    }).value()

(example from underscore/lodash)

The problem with this is to be able to add a custom function to this pipeline, you have to extend the prototype of the library with your own functions, or re-implement it.

Extending the prototype is considered bad practice for a lot of reasons, for example due to polluting code in other modules or upstream changes breaking your code.

If instead of chaining methods we chained functions we are left with this awkward syntax:

    custom(filter(map(compact(array), function(x) {
         return x * x;
    }), function(x) {
         return x > 100;
    }));

it's much cleaner to write it like this:

    compact(arr)
    |> map(function(x) {
        return x * x;
    })
    |> filter(function(x) {
        return x > 100;
    })
    |> custom

The linked page has a nice example and explanation about this concerning promises https://github.com/mindeavor/es-pipeline-operator#sample-usa...

rakoo• 11 years ago

The problem with this is to be able to add a custom function to this pipeline, you have to extend the prototype of the library with your own functions, or re-implement it.

That's exactly what transducers try to solve, and fortunately they do exist for js (http://jlongster.com/Transducers.js--A-JavaScript-Library-fo...) where processing of individual elements is separated from plumbing, so you can add your own custom computation in the chain. It even has pretty good performances (http://jlongster.com/Transducers.js-Round-2-with-Benchmarks), all with the standard data structures !

Of course transducers are useful only when you want to transform data, not when you want to act on them in the chain.

wnfewnlkew2• 11 years ago

Please no.

Javascript's beauty lies in its versatile simplicity. Can we just leave these sorts of things to systems programming languages like C++? I came to javascript because of its lack of cruft, but if libraries start adopting this then I'll be forced to put it in my code as well. If that happens then I'll probably leave for nim or clojure, though that's not preferable.

edgyswingset2• 11 years ago

Can't speak for nim, but Clojure has a near-exact feature which is used all over the place, the thread-first and thread-last macros.

Example:

    (defn do-stuff [params-map]
       (->
          (build-object params-map)
          (do-thing-with-object)
          (extract-thing)
          (format-thing)))

I'm not sure why you'd choose Clojure to get away from things like |>.

wnfewnlkew2• 11 years ago

Clojure is a functional language, so it meshes well with the rest of the syntax, but in javascript's case it's just syntax bloat. It's going to be really jarring in the middle of regular imperative code, and it's not going to be too far removed from

  cout << "foo";

bpicolo• 11 years ago

Elixir has this exact feature. I'm sure it's not the first language to either.

abrezas2• 11 years ago

C++ has a piping operator?

toni2• 11 years ago

    attrs
        |> bounded('age', 1, 100)
        |> format('name', /^[a-z]$/i)
        |> Person.insertIntoDatabase

This reminds me of those "Perl Poetry" code snippets from 2001.

mbreese2• 11 years ago

Yes, and we all know how that turned out with Perl. I don't think Perl is necessarily a good model for readable code.

This seems to me to be a case of "just because we can, doesn't mean we should."

gotchange3• 11 years ago

This is syntactic sugar not cruft. They're not proposing to add any new functionality to what we already have.

jordz• 11 years ago

As an F# user, I like this a lot, very similar although it can confuse people. It's very much a trait of functional programming and it's easy to read once you get the hang of it :)

radicalbyte3• 11 years ago

That's beautiful, the linear pipeline better matches your intuition about the computation than the nested functions.

Can we have this in C# 7? :)

danbruc4• 11 years ago

This essentially exists - extension methods.

radicalbyte3• 11 years ago

Extension methods only apply to instances and they must be declared explicitly.

This syntax can be used anywhere as long as the types signatures of the functions are conducive to the chaining.

wycats8• 11 years ago

If we're going to do this operator, I would prefer a regular arrow:

    import { sortBy, filter, map } from 'array-like';

    let names = document.querySelectorAll('.entry')
      -> sortBy('title')
      -> filter(entry => entry.getAttribute('data-url') in whitelist)
      -> map(entry => entry.getAttribute('name');

I personally find this more readable than either the bind proposal (which looks too much like a property access, and not enough like a function call, to me) and the proposed pipeline operator, which doesn't have an immediate meaning to my eyes.

wycats8• 11 years ago

That said, the main counterargument to this proposal is that it's already possible to get "left to right" composition in userland without this operator, as underscore shows.

The main counterargument to "do it like underscore" is that underscore still ends up being an object whose methods are added to a prototype, not imported.

The benefit of the "pipeline" or "bind" approach is that you get left-to-right composition while still importing the names you're using.

vruiz• 11 years ago

I personally find this more readable than ... the proposed pipeline operator, which doesn't have an immediate meaning to my eyes.

Really? have you never seen unix pipes?

wycats8• 11 years ago

I would find pipes to be very readable (but the single pipe operator is taken). The triangle isn't immediately an analogy to "unix pipes".

tallerholler2• 11 years ago

that won't play nice with coffeescript!

wycats8• 11 years ago

CoffeeScript should have thought of that ;)

danbruc4• 11 years ago

Why the arrow? That works just fine with a dot.

  let names = document.querySelectorAll('.entry')
                      .sortBy('title')
                      .filter(entry => entry.getAttribute('data-url') in whitelist)
                      .map(entry => entry.getAttribute('name');

wycats8• 11 years ago

Only if those methods are added to NodeList, which they haven't been.

(and of course 'sortBy' is nonstandard)

pygy_2• 11 years ago

I would prefer a regular arrow

And I want it painted green ;-)

... which doesn't have an immediate meaning to my eyes.

It's a matter of background. F#, Julia and Livescript have identical operators, likely among others.

https://msdn.microsoft.com/en-us/library/dd233229.aspx#Ancho...

http://docs.julialang.org/en/release-0.4/stdlib/base/?highli...

http://livescript.net/#piping

wycats8• 11 years ago

Fair enough.

I personally consider compound operators (asymmetrical) to be pretty risky (the do, indeed, depend on background), and I like '->' because it has a clear meaning outside of programming.

But you're right, this is a bikeshed.

I'd prefer either version of the pipeline operator to bind.

msoad• 11 years ago

an Array.from will make it possible to do it like regular arrays. I don't see a good use case for this operator really.

XorNot• 11 years ago

I don't entirely understand where this gets me that promises don't.

danneu3• 11 years ago

Promises are an abstraction for managing asynchronous operations.

This proposal is about function application.

Animats• 11 years ago

The single-argument case is elegant, and the multiple-argument case is awful.

    var newScore = add(7, validateScore( double(person.score) ))

becomes

    var newScore = person.score
      |> double
      |> score => add(7, score)
      |> validateScore

This is very Forth-like. Operands get pushed on the stack, and operators take from the stack and push results back. In Forth, there's no operator precedence and no parentheses; it's pure reverse Polish.

Why not go all the way to pure RPN?

     7 person score . double ValidateScore add newScore =

See how it simplifies the syntax? Of course, you have to know how many arguments each function takes.

Do we really want to go down this route? It's all syntax; it doesn't add any capability. As an extension to an existing language, it's likely to produce a mess.

Mixing functional and imperative notations is a problem. LISP is all nested parentheses, and that bothered a lot of people. Forth eliminates all the parentheses, which bothered different people. Python was pure imperative, ("lambda" went in over major objections) and is gradually getting more functional features, but they don't fit the syntax well.

simplify• 11 years ago

It's not just syntax; it actually opens new ways for API design. See abreza's comment: https://news.ycombinator.com/item?id=10686596

Sonata• 11 years ago

After having used this operator in Elixir and Elm, I would love to see it in JS. I know it seems like a superficial issue, but I think the left-to-right chaining flow is one of the main things people miss as they move from an OO style to a more functional one.

gotchange3• 11 years ago

It is not superficial or trivial at all. This proposal helps to ease the cognitive load required when reading or debugging code at least for the straightforward cases and makes writing code even more interesting and intuitive.

pygy_2• 11 years ago

@dang, was this manually penalized? Or is it a false positive to the flamewar detector?

It looked like a fine submission.

52 points, an hour ago, and it lies on the third page...

mateuspv• 11 years ago

If is to be more functional like e prefer add functions: compose and curry:

1 - // With pipe: var result = "hello" |> doubleSay |> capitalize |> exclaim;

  With compose:
  var greet = compose(exclaim, capitalize, doubleSay);
  greet("hello");
  // or compose(exclaim, capitalize, doubleSay)("hello");

2 - var person = { score: 25 };

  // With pipe:
  var newScore = person.score
    |> double
    |> score => add(7, score)
    |> validateScore

  // With compose and curry:
  var newScore = compose(validateScore, curry(add(7), double);
  newScore(person.score);

3 - Mixin paradigms

  // With compose and curry :
  var newScore = Function.compose(validateScore, add.curry(7), double);
  newScore(person.score);

ES7 Proposal: The Pipeline Operator

Related Stories