Pipe: Infix syntax for Python

http://dev-tricks.net/pipe-infix-syntax-for-python

by toni • 15 years ago

240 41 15 years ago

dev-tricks.net

biot2• 15 years ago

This reminds me of C#'s LINQ, but for Python... PLINQ?

http://stackoverflow.com/questions/2062540/help-with-linq-ex...

mmavnn2• 15 years ago

He mentions in the link that linq is an inspiration.

biot2• 15 years ago

Thanks, I had missed that in the comments.

darklajid• 15 years ago

This reminds me more of the |> 'operator' in F#, doing the ~very same thing.

draegtun• 15 years ago

And the feed operator in Perl6: http://perl6advent.wordpress.com/2010/12/10/day-10-feed-oper...

mmavnn2• 15 years ago

Very nice piece of sugar. Pipes are one of the things I love about f#, and it's nice to have a quick and easy implementation for when I'm playing with Django/App Engine.

va1en0k• 15 years ago

that's great! I always wanted something like this for Python

the problem is, I rarely feel like I can just use all these cool-syntax libraries for production, since they don't add a lot of value. why do I need a whole dependency dependency for something that just make one or two line more readable?

I've written a syntax enhancer for PHP once, but it just added one feature (array dereferencing) for a small but noticeable price of setting up; I never considered it for production usage because there weren't enough features to justify it. (CoffeeScript is an example of justifiable price for huge changes and multiple ideas)

for personal just-for-fun projects (like solving project euler problems) it's an awesome idea, though

JanezStupar• 15 years ago

Well, the way I see it - these tools open up whole new array of possibilities of writing code in a more intuitive fashion.

This may enable a completely different and cleaner architecture or a mess. Probably as always it starts with a mess and results in clean along the way once you resolve all the problems you should know about but didn't.

Key point is that this is exactly the way our languages and industry evolves. jQuery also started as someones just-for-fun project and poised risk (and still does) to all the projects that used it.

In reality a person needs to go and just use stuff like this, with experience you learn to see if something works or not.

djacobs• 15 years ago

The way I code Python, this has a real potential to clean up any codebase I'm working on in a major way. (I've complained several times about the need for intuitive chaining in Python.)

ohyes• 15 years ago

What I would like to see is a way to separate the the syntax of my code from the semantics of my code. Ideally, my code editor should be able to take the same piece of code and display it in a few different styles of syntax.

Then I could write my code in whatever syntax I want, and someone else could flip a switch and change it into their preferred style of syntax.

I am honestly kind of puzzled as to why the concept of MVC disappears as soon as we start writing coding tools. It seems that for any reasonable syntax that you can write a parser for; it should also be possible to write an emitter from an 'intermediate' syntax tree.

asharp• 15 years ago

From what I remember, the problem isn't producing something that parses (for the emitter), the problem is producing something that could still be considered human readable and/or not horribly ugly.

wladimir• 15 years ago

I've also thought about this a few times. It's a pity that the SCID (Source Code in Database) paradigm never really caught on. It would make things such as representing the same code with different syntax (depending on user preference) possible. It also makes refactoring/renaming easy as names are only stored in one place, the rest refers to the same 'entity'. And you're right, best practices for data formats are applied everywhere except for code. After all, we're now in 2011 and still using old-fashioned text files.

gerner• 15 years ago

I've worked lots of smart engineers who use these sort of "just-for-fun" tools, or simple hacks. At the time I often think exactly as you do, "why do I need a whole dependency..." And often I leave feeling the same way.

But more than once-as simple as putting null before equalities in conditionals, or as complex as setting up squid to cache just robots.txt for a crawler-I've been surprised and glad that I'm working with that quirky dude who insists on paying a small price for something useful. And I find myself using it again in the future because it actually gave me real value, made me and my team more productive, or happier, and hopefully made our product better.

pepijndevos• 15 years ago

Server is unresponsive. Maybe Coral will cache it sometime: http://dev-tricks.net.nyud.net/pipe-infix-syntax-for-python

toni• 15 years ago

github repo: https://github.com/JulienPalard/Pipe/

ivank• 15 years ago

Possibly inspired by Pypes: http://arvindn.livejournal.com/68137.html

aufreak3• 15 years ago

Remove the pipe symbols and you get Forth syntax!

srean• 15 years ago

Coincidentally enough I was looking at http://code.google.com/p/python-pipeline/ a few days ago. I fully expected them to be related by a fork, but they dont seem to be. Python-pipeline executes a similar idea, but to infix-chain iterators by way of generators. I guess some abstractions are just too handy and useful and the chances are high that they will be implemented/discovered independently. Pipe is just one of them.

d0m• 15 years ago

Is there any helper function that convert existing functions into pipe's style?

For instance:

  [1,2,3,4,5] | p(sum)

where p would transform a def(seq, args, kwargs) to def(args,kwargs) and yield the results?

Also, maybe:

  [1,2,3,4,5] >> sum

where that could be made automatically ?

heyitsnick2• 15 years ago

Keep in mind I have no idea how this magical code works. In fact I can't even find the pipe character on my keyboard right now (I am copy and pasting it!). But this seems to work:

    @Pipe
    def p(iterable, function, **kwargs):
        return function(iterable, **kwargs)



    In [3]: from pipe import *
    In [4]: [1,2,3,4,5] | p(sum)
    Out[4]: 15

yesbabyyes• 15 years ago

Or even just wrap the function in Pipe:

  >>> from pipe import *
  >>> [1, 2, 3, 4, 5] | Pipe(sum)
  15

jerf2• 15 years ago

"In fact I can't even find the pipe character on my keyboard right now"

On standard US keyboards, it's shift-backslash. YLMV.

Fice• 15 years ago

Reminds me of the threading macros (-> and ->>) from Clojure.

geophile2• 15 years ago

I have built something similar, coming at the problem from a different direction. I wanted a Unix-style shell, but piping objects instead of strings. For example, to sum x^2 from the command line, x = 0-9:

    bash$ osh gen 10 ^ f 'x: x**2' ^ red + $

^ is the pipe symbol, red means "reduce", $ means output, and f means apply the given function.

The same capabilities are available as a Python library:

    from osh.api import *
    osh(gen(10), f(lambda x: x**2), red(lambda x, y: x + y), out())

    from osh.api import *
    osh(gen(10), f(lambda x: x**2), red('+'), out())

Osh can also run results on multiple hosts at once, in parallel, combining the results in various ways; and integrates database access, piping tuples to and from sql commands.

http://geophile.com/osh

iki23• 15 years ago

Nice! Would deserve a github repository and keeping the pypi pakcage up-to-date: http://pypi.python.org/pypi/osh/

geophile2• 15 years ago

Yes, the pypi entry is obsolete. For some reason, I get into some sort of endless lost username/password purgatory whenever I try to register there. I really should bring that up to date and put the package in github.

jhuni• 15 years ago

Infix notation is more readable then poorly written "classical" prefix notation.

algorias• 15 years ago

I wonder about the usefulness of this. Doesn't the python generator syntax take care pretty well of most use-cases?

The first example could have been written like this, which is much simpler and easier to read than both the prefix and infix versions:

sum(x * x for x in take_while(fib(), lambda x: x < 1000000) if x % 2)

lloeki3• 15 years ago

Inserting/disabling/swapping intermediate processing steps in between is immensely easier with pipes, and you don't have to take care of balanced parentheses.

xtacy• 15 years ago

A similar question was asked on Stackoverflow:

http://stackoverflow.com/questions/4961017/clojure-style-fun...

Some answers seem enlightening:

  While I sympathize with the desire to create cool new language constructs (à la Lisp macros), it is not really the Python philosophy to do this:

    >>> import this
    [...]
    There should be one-- and preferably only one --obvious way to do it.

scott_s• 15 years ago

Hey, fogus and btilly! Small world.

riobard• 15 years ago

The Fibonacci generator hurts my eyes! Here is a cleaner version.

    def fib():
        a, b    = 0, 1
        while True:
            yield a
            a, b    = b, a+b

Anyway, the order of the pipeline-like operation in Python sometimes really bugs me. This one addresses part of it.

Recently I'm playing with Scala and it is really neat to do these stuff:

    // A lazy Fibonacci generator (a Scala Stream)
    lazy val fib: Stream[Long] = Stream.cons(0, Stream.cons(1, 
      fib.zip(fib.tail).map(pair => pair._1 + pair._2))) 

    fib filter(_ % 2 == 0) takeWhile (_ < 4000000) sum
    // 4613732

moomba• 15 years ago

Thats great, now if they added some of the other lisp features (macros, etc.) I'd consider using python.

zacharyvoase• 15 years ago

I wrote a really similar library called Calabash a short while ago: https://github.com/zacharyvoase/calabash

It’s all fully tested (python setup.py test), and I really like using it, but I haven't done so in any large-ish production app yet because it just looks completely different to the surrounding code. I have used it with success in some small scripts, but that's about it.

jerf2• 15 years ago

ISTR that when people discussed various ways of overloading the pipe on comp.lang.python back in the day, one thing you have to be careful of is that the bitwise OR operator binds relatively weakly [1], and that can result in some surprising behavior if you intuitively assign the | a precedence in your head more suited to the behavior implemented here. In this particular case, though, you probably wouldn't hit this problem very often... though in some ways that can make the problem worse in its own way.

[1] http://docs.python.org/reference/expressions.html#summary

lloeki3• 15 years ago

Mentally making a note of that. I was wondering what kind of side effects the use of | would end up having, if any.

VMG• 15 years ago

This calls for allowing the entirely user-defined $-operator in python 4.

nyellin• 15 years ago

What if two Python-based frameworks use $ for separate purposes? (e.g. Django and NumPy)

OnesimusUnbound• 15 years ago

Nice source code, readable, and well documented. The author uses generators when it makes sense.

I try to use ipython and pipe as my shell to see how it feels.

Pipe: Infix syntax for Python

Related Stories