Functional programming in Python

http://docs.python.org/release/3.1.5/howto/functional.html

by plessthanpt05 • 14 years ago

133 61 14 years ago

docs.python.org

pepijndevos• 14 years ago

What is really missing is a set of decent data structures.

jlarocco• 14 years ago

What do you mean? What would you add on top of the built-ins and stdlib data structures?

swannodette4• 14 years ago

I believe he is talking about persistent data structures.

andreasvc4• 14 years ago

There are situations where balanced search trees are appropriate. A general purpose priority queue (with delete and decrease-key operations); there's actually code for this in the _documentation_ of the heapq module; this seems really odd to me, why not just include it? It's also a shame that heapq is built on list instead of being a first-class data-structure, it feels bolted-on. Bitwise tries would be nice as well.

chimeracoder8• 14 years ago

I doubt you'll see that - at least in CPython. Guido's been very clear about how he feels about implementing functional paradigms in Python, so it will probably never support functional programming in the standard implementation - just a bit in syntax/interface.

soc883• 14 years ago

I honestly don't understand that. Data structures are just code ... isn't it much more work to change syntax and interfaces instead of adding some decent implementations of data structures to the library?

soc883• 14 years ago

I agree completely. It is surprising how many Python developers are unfamiliar with it.

theorique• 14 years ago

This is a useful reference but not really all that current. It dates back to 2006 - the functional constructs in Python now have evolved and changed over the years.

chimeracoder8• 14 years ago

One thing that's been key for me is namedtuple (in the collections module). It's immutable like a tuple, but the values can be accessed by name just as if they were object attributes built with the class keyword. It's great for creating generic functions (think Lisp and CLOS) instead of using Python's prototypical system. And since tuples can contain any objects and functions are objects, you can bind callables like lambdas to those names, which allows you to have 'methods' if you want as well.

That said, the standard library leaves something to be desired for the functional style. It makes sense for object-oriented programming to have non-mutating functions return the result and mutating functions return None (think sorted vs. sort), but this can get rather irritating when you're trying to write in a purely functional paradigm.

Also, am I forgetting my Python, or does it not have a great solution for appending to the front of a list? One thing I love about Lisp is that it's ridiculously easy to write (cons item some-list), or even (cons item1 (cons item2 some-list)). Doing the same thing in Python is irritating, because insert() doesn't return the result list.

...or maybe that's what I deserve for trying to write Lisp-like code in Python, anyway....!

EDIT: As noted in the comments, the '+' operator will suffice here. Though since lists are really arrays and not linked lists, this functional way of thinking will result in horribly inefficient CPython code.

aeter• 14 years ago

How about [item] + [some, list]?

sureshv4• 14 years ago

That works but it's kind of a pain that the natural pythonic syntax: ['a'].extend(some-list) doesn't return a list.

chimeracoder8• 14 years ago

Hmm, I forgot that '+' was overloaded for lists in Python - nevermind that last statement, then.

The rest still holds, though - not all functions have a mutable and immutable counterpart (like sort) and map() returning lists makes them rather unwieldy. I think they fixed this in Python 3, though I'm not sure if it was backported to 2.7.

emiliobumachar• 14 years ago

Maybe there's no way in the standard library, and it may be a valid point that it should be there, but you could trivially define your own 'cons' function and write cons(item, someList)

TazeTSchnitzel• 14 years ago

namedtuple is awesome, until you realise it works internally by using eval and generating the source code of a class.

chimeracoder8• 14 years ago

And Lisps work internally by generating lists and evaluating those, but you don't see me complaining about macros!

In all seriousness, though, if I really wanted a functional implementation, I wouldn't be using Python. It's more just nice to know that I can create an object that's immutable and which functions more or less like a CLOS object, so that I can think in that mindset instead. Also, I believe it doesn't come with all of the hidden functions that it would otherwise.

In the end, if I really cared, I'd actually write a wrapper around the type() function.

mattdeboard• 14 years ago

namedtuple is still awesome. The fact that it's eval'ing a string to initialize it doesn't make that any less the case.

However, I'll agree that it would be disingenuous to say it's not a little peculiar, and surprising to see the source for it :)

slig• 14 years ago

Why does it do that, instead of some other more pythonic way?

benhoyt• 14 years ago

See also http://bugs.python.org/issue3974, where someone submits a patch for namedtuple that doesn't use exec. The maintainers reject it as less clear and probably slower. I agree there is "nothing unholy in using exec", though I found this weird at first too.

samdk• 14 years ago

Python lists are array-backed, Lisp lists are singly-linked lists. Appending to the front of an array list is O(n)--you don't want to do it. Appending to the back is constant time, but is an in-place operation and doesn't return anything, which is less than ideal for FP purposes.

chimeracoder8• 14 years ago

That's the problem. I can fake a functional paradigm within Python in some settings, but it falls apart at the seams.

andylei2• 14 years ago

if you want to append to fronts and backs of lists, you should use deque: http://docs.python.org/library/collections.html#collections....

FuManJew• 14 years ago

pronounced "deck"

rbonvall• 14 years ago

    One thing I love about Lisp is that it's
    ridiculously easy to write (cons item some-list),
    or even (cons item1 (cons item2 some-list)).

Iterators can help (and in O(1)):

    >>> from itertools import chain
    >>> some_list = range(5)
    >>> item = 99
    >>> a = chain([item], some_list)
    >>> for x in a:
    ...     print x
    ...
    99
    0
    1
    2
    3
    4

gbog4• 14 years ago

Yes. That's the hole point of the article, and matches very well with my understanding of SICP: functional programming is possible only if you model changing state in streams, which are just called iterators in python. All the fuss about python missing some parts too be really functional seem to have missed the point to me.

dustingetz• 14 years ago

as has been pointed out, python isn't particularly great at doing hardcore functional programming due to lack of native persistant datastructures. however i've found it great for learning functional programming without having to get used to the syntax of real functional languages.

here are some different ways to implement functional sequence operations without native python syntax like `yield`:

https://github.com/dustingetz/sandbox/blob/master/etc/lazy.p... https://github.com/dustingetz/sandbox/blob/master/etc/map.py

and an unfinished, sloppy impl of monads in python with lots of minor errors:

http://www.dustingetz.com/2012/04/07/dustins-awesome-monad-t...

andreasvc4• 14 years ago

What do you mean with lack of persistent data-structures? There are immutable data-structures such as numbers, strings, tuples, and frozensets (unfortunately no frozen dicts) -- seems plenty to me. Or do you mean the technical meaning of persistent data-structures, as in one that gives access to all previous versions?

swannodette4• 14 years ago

A good set of persistent data structures would give you all the generality, utility, and performance of Python data structures like lists and dicts and none of the mutability.

technomancy• 14 years ago

I would have thought the broken statement/expression dichotomy would be a bigger annoyance, though I've never used Python in anger myself.

Rickasaurus• 14 years ago

I agree so much. Real functional programming requires an expression oriented language.

SiVal• 14 years ago

I was at a Python user group meeting with Guido a few years back, and I asked him about his plans for functional programming support in Python. He responded that he thought functional programming was a lot less useful than all the hype would suggest, and that, far from planning more support for functional programming, he regretted having put in as much support as he already had. Ideally, he said, he would remove things like lambda instead of making them stronger.

Even things like tail call elimination, he said, would just encourage people to use more recursion, when iteration was easier to read, and in places where TCE would work, the programmer could obviously convert it to iteration himself and should do so for code clarity.

I like Python, and I like a lot of things about functional programming, so it wasn't the answer I was hoping for. Unless Guido has had a change of heart since then, I wouldn't pin my hopes on Python ever becoming a first-rate functional programming language. It's first rate at the things Guido values.

gbog4• 14 years ago

I don't know. It seems to me, from reading SICP, that functional programming doesn't mean lambdas or map reduce, it means pure functions as the main programming unit, instead of object or instruction or declaration. Then it is possible to write functional code in python without lambdas, map, using streams (iterators) as the state keeping data structure.

olavk2• 14 years ago

Python has added list and set comprehension syntax, which are just as functional as eg. map and filter. So I don't think it is functional constructs in general he is against, but rather specifically the lispy idioms with map, filter etc. with functions/lambdas as arguments.

Decorators are another example of functional programming added to Python, but with a pythonic syntax.

Python supports both imperative, OO and functional idioms, but generally only prefers one of them for a specific task. For example loops are preferred to recursive functions when appropriate. Objects are preferred to closures for mutable state, so mutation of captures variables has not been supported (support have been added recently with nonlocal, but it has clearly not been a priority).

Python is interesting because it is multiparadigm while trying to avoid fragmentation where you can solve the same problem in totally different styles.

ericmoritz• 14 years ago

I wish he would add TCE for the sole fact that parsing tree like structures feels much more natural using recursion than it does with a for loop. I am always paranoid that a really deep structure will cause an stack depth exceeded exception.

There's a quote from Guido about private attribute access where he said, "We're all adults", and that leaving the open is beneficial.

Recursion is another tool that he should let us adults use if it solves a problem better and TCE is lovely feature to have. Frankly, if you're using recursion, you either just learned it in a CompSci class or you know what you're doing. Either way, daddy Guido shouldn't tell us no :).

craigyk• 14 years ago

I don't remember where I read it, but "watching someone try to do functional programming in python can be painful". I have to agree quite a bit with that. The distinction between lambdas and functions, expressions, etc.

I started doing a lot of programming in CoffeeScript a few months ago, and between the JS underpinnings and the CS syntax you can do things that would just be painful in Python.

Functional programming in Python

Related Stories