Monads in Python (with nice syntax)

http://www.valuedlessons.com/2008/01/monads-in-python-with-nice-syntax.html#6799472260664409462

by llambda • 14 years ago

92 48 14 years ago

valuedlessons.com

hxseven• 14 years ago

BTW: This was already submitted to HN.

http://www.hnsearch.com/search#request/all&q=Monads+in+P...

But that doesn't make it less interesting ;)

temac• 14 years ago

This is very interesting, very intelligent. Indeed this is too much intelligent. There is a huge amount of highly valuable programmers that won't be able to understand all that mess, and the gain is questionable at least given that there are alternate constructs that are way better compromise between expressiveness and maintainability.

Use the right tool in its intended way: don't try to retrofit constructs in languages were they don't fit, just for an accessory characteristic. (Every line of an imperative language is conceptually kind of a monad, so the article here is just trying to convince us that yet another form of hidden goto is good, by an ad-hoc extension in a language were more than half of monad necessity is missing)

beering• 14 years ago

It might require an intelligent programmer to create, but as a library, can be used by regular, unintelligent programmers. If he documents his monads well and shows how to use it, and it's effective, then it's not such a big deal.

Or, you know, this is more of an attempt to show programmers stuck in simpler languages what other PL features have to offer.

dllthomas• 14 years ago

"Every line of an imperative language is conceptually kind of a monad[.]"

Every line of an imperative language is conceptually in the IO monad. That doesn't mean that there's never reason to run things in another monad - many do things IO can't.

crucialfelix2• 14 years ago

The problem with Monads is that the people who understand them are the least qualified to explain them to other people.

dons5• 14 years ago

This is specious. Many people understand monads, and their ability to teach and explain varies wildly.

ufo5• 14 years ago

I disagree. I think the only big problems with teaching monads is that they are a mathematically inspired abstract interface (while most people only need to bother about the concrete implementations of this inteface) and that Haskell forced them too soon on people, since you need monads to do IO.

StavrosK2• 14 years ago

The corollary would mean that the people who can explain well can't understand monads, which means that some people just can't understand them. We're doomed!

ibotty• 14 years ago

not that i agree with the top comment, but your reasoning only works for a static view of people (i.e. they cannot learn something and at the same time forget something).

freework• 14 years ago

This is missing the one thing that all other "let me explain monads to you" articles is missing: a real example of what problems monads solve. As far as I know, a monad is some kind of mathematical construct that is very useful when solving mathematical proofs, but has no use for the every day programmer. The examples in these kinds of articles are always completely abstract. You can very easily describe a database by using the "customer/phone number" example, yet I've never seen such an example with monads.

dons5• 14 years ago

http://www.haskell.org/haskellwiki/Monad#Interesting_monads

Nothing to do with proofs.

Monads are a generic interface -- an API -- for changing up the computational rules of your programming language.

Just think about that for a second -- its a big deal.

Maybe you want to work with non-deterministic threads, or with continuations, or quantum computations, or backtracking search, or exceptions or partial results, or some other notion of computation. But you want to still write in your usual language.

So you embed that computational style as a monad, and now you have a really nice API for programming in that different environment -- inside your everyday language.

Its simple, but profound. With (good support for) monads you aren't constrained to the default computational rules of your programming language anymore.

You are free.

lhnz• 14 years ago

The problem with this description is that I don't know what kind of benefits I can get from using different computational rules... Can you show me?

andrewcooke• 14 years ago

well, it's not claiming to explain monads.

but it does contain a pile of examples. try looking at the mailbox example and then running the code. you'll have a pretty powerful impression of the way that monads can modify apparent control flow semantics.

dbaupp2• 14 years ago

It's explicitly not explaining what a monad is, first paragraph of the article:

I've been informed that I didn't clearly explain what monads are and what the code examples are supposed to do. Sorry. I guess I assumed too much preexposure to monads. If you're no familiar with them, there are already so many tutorials on monads that I'll simply direct you to two of my favorites: spacesuits and wikipedia. I've also added more explanations of what the code snippets are supposed to do. I hope that helps.

(The spacesuits link is dead (live[1]) and neither directly address "your motivating example" point, although the non-monadic vs monadic code comparisons demonstrate that monadic code can be much much shorter and less repetitive.)

[1] http://web.archive.org/web/20081206204420/http://www.loria.f...

Evbn3• 14 years ago

The spacesuit one has been explicitly disowned by its author. "Programmable semicolons" is the rough modern consensus on the best one-line metaphor.

Strilanc• 14 years ago

I give an example in "How would I even use a Monad (in C#)?" ( http://twistedoakstudios.com/blog/Post867_how-would-i-even-u... ), assuming a hypothetical extension to the language:

    ///<summary>Pulls the monad type outside of the list type by binding across the list's elements.</summary>
    ///<remarks>Not actually possible to compile in C# today.</remarks>
    public static M<IEnumerable<T>> BindAll<M, T>(this IEnumerable<M<T>> sequence)
        where M : Monad { //<-- Type parameter M takes a type parameter T. Not currently possible.
        return sequence.Aggregate(
            M.Return(Enumerable.Empty<T>()), //<-- Static interface method. Not currently possible.
            (wrappedAccumulator, wrappedNextItem) =>
                from accumulator in wrappedAccumulator
                from item in wrappedNextItem
                select accumulator.Concat(new[] { item }).ToArray().AsEnumerable());
    }

This single method combines several apparently distinct methods: waiting for all tasks, choosing all of the combinations from a list of lists, etc.

ufo5• 14 years ago

> a real example of what problems monads solve

Have you ever used an async promise library in Javascript? Its a monad! (you have a function to create a new "async promise" given a sync value and you have a method to chain a promise with a callback that returns a new promise)

The complicated thing about monads is that they are an abstract interface. Having the abstract interface is good if you want to write generic code that works on all monads (and if you want to prove things about your code) but makes it hard to understand stuff. Another problem with monads is that its API can't be implemented directly in most languages so they get restricted to the more functional languages: one of the methods (>>=) receives an anonymous function as an argument (and many languages don't support those)and another method (the awkwardly named "return") is polymorfhic on its return value (OO languages can only handle polymorphism on the first input argument, the "this").

Evbn3• 14 years ago

With templates/generics, polymorphic in return type is no problem. Consider Java Guava's

    Lists.<String>newArrayList()

mrcrassic• 14 years ago

I think one of the best and clearest examples of a useful monad is the Error monad in OCaml (and Haskell, I think). Error is a type that can return two different things depending on the result of a computation. One could use exceptions for this sort of thing, but this is much, much cheaper.

dustingetz• 14 years ago

i've worked through this article a couple times and his implementation never really sat well with me, the "with nice syntax for monad comprehensions" bit really obfuscates the implementation.

here's python source to a monadic lisp interpreter, which i wrote to follow along the paper "monad transformers and modular interpreters" [1]. i think this is a much simpler implementation of python monads than provided in the ValuedLessons article. https://github.com/dustingetz/monadic-interpreter

Study of this implementation will teach you why nobody actually uses monads in python for non-toy projects. A literal port of this code to clojure would feel so much more idiomatic and not hacky at all.

here are some smaller code dumps demonstrating the fundamental concepts to that monadic lisp interpreter:

http://www.dustingetz.com/2012/10/02/reader-writer-state-mon... http://www.dustingetz.com/2012/10/07/monads-in-python-identi...

[1] http://haskell.cs.yale.edu/wp-content/uploads/2011/02/POPL96...

dbaupp2• 14 years ago

I think the point of this article is to make a "nice" syntax for monadic operations in python (i.e. avoiding all the lambdas and extra parens in the "bind(mx, lambda x: my)"/"mx >> (lambda x: my)" pattern), no matter how unclear and non-pedagogical the implementation ends up.

almost2• 14 years ago

Can anyone explain to me how the continuation Monad is working there? As far as I know there's no way to return from the same yield twice, which is what a callcc would need to do. I'm going to carry on trying to puzzle it out but a pointer would be helpful!

EDIT: I just tried out the code and it doesn't support multiple returns. Isn't that pretty much the thing that defines the Continuation Monad or am I just not getting it?

qznc• 14 years ago

Technically, it is still a continuation, even if it does not match the behavior of Haskell's continuation monad.

http://c2.com/cgi/wiki?SingleUseContinuation

almost2• 14 years ago

I didn't know that, thanks. Seems less useful though.

What's the correct term for a continuation as I understand them? (eg: as they are in Haskell, Ruby, Scheme etc)

klibertp• 14 years ago

I went through this article some time ago and, while it's certainly neat implementation and worth understanding (that was the first recursive generator that made sense to me - thanks to this I had much easier time understanding Scheme's call/cc and it's uses), it needs to be noted that it is not a generic monad implementation. You can not, for example, implement list monad (I tried very hard and couldn't, I saw the same said in comments below the article, but if it's possible please let me know!) using this implementation and syntax, because you can not "rewind" Python's generators.

So, while it's nice implementation for some kinds of monads it's not nearly general enough to be (IMHO) called "monads in Python" - it is possible to implement fully general monads in Python, but you need nested lambdas and there is no getting away from it.

Still, very nice hack and worth spending an evening (or two) to understand how it works.

andymatuschak• 14 years ago

This approach to the "do" syntax requires that the function implementer know the monad type a priori.

That defeats some of the purpose of monad as general abstraction. One of the big ideas here, as far as I'm concerned, is that monads represent an abstraction over a common structure of programs. Lists, Maybes, threading models, etc, all share this same language of structuring computations. This is significant because it means that one can build functions which manipulate those structures, and then those structures can be deployed generically. Depending on how you're counting, that means third-order abstraction.

For instance, consider liftM from Control.Monad:

  -- | Promote a function to a monad.
  liftM   :: (Monad m) => (a1 -> r) -> m a1 -> m r
  liftM f m1              = do { x1 <- m1; return (f x1) }

This represents a common idea across e.g. both List and Maybe. I've got a function of values. I want to turn it into a function of Lists / Maybes. Well, liftM recognizes the similarity of List's and Maybe's structure and provides a single implementation suitable for both places.

The decorator described in this article cannot function at liftM's level of abstraction: at least as far as I can tell (as a non-Python programmer), it requires a monad type parameter. If that's true, I think it's missing a key idea behind the power of monads.

ufo5• 14 years ago

Not really. Python doesnt have typeclasses so liftM would be a fucntion receiving the monad dictionary as an explicit parameter. At that point, you can pass that dictionary to the decorator:

    def liftM(m, f, x):
        @do(m)
        def doNotationBody():
           ...
        ...

csense• 14 years ago

The first line of his first example of using his monad implementation is:

@do(Maybe)

Ctrl+F Maybe. It's not defined anywhere!

I was excited to see that someone could maybe finally explain to me, in Python, why people are so excited about Monads.

I looked at Haskell once. It looks impossible to understand without understanding Monads. (And near-impossible to understand even if you do understand Monads -- the syntax is a nightmare.) I thought maybe this article would help me, by using the syntax of language I do understand -- Python -- with a concept I don't understand -- Monads.

But this excitement was misplaced, since this isn't a usable demo. Please, if you're the author of this article or someone who understands this code, post a Github repo, or a tarball on your HTTP server, or something with a runnable, simple example of code that uses the article's approach to monads.

Monads in Python (with nice syntax)

Related Stories