"On iteration": Why people leave Python

http://brehaut.net/blog/2010/on_iteration

by mattdw • 16 years ago

59 80 16 years ago

cgranade• 16 years ago

That article makes some really good points, but I disagree. To me, Python is quickly becoming a good middle ground where I can use these powerful functional ideas, then drop down to imperative whenever I need to. In particular, this means that I can use Python to do scientific computing or to easily work with native libraries while still maintaining a lot of the functionality of more pure languages.

pgbovine2• 16 years ago

agreed ... also i think the dynamically-typed nature of python (and similar languages) makes them very amenable to rapid prototyping and experimentation (code just runs without having to fight the type inferencer or checker). python lets you do clean and beautiful functional programming, but also, as you mentioned, to drop down to quick-and-dirty imperative mode

the author has a good point about pure functional languages being better-suited for concurrent programming, though

brehaut4• 16 years ago

Just to be clear, Clojure (and most lisps) are also dynamically typed and thus allow just the same experimental approach. Clojure and Common Lisp (I can't speak for other lisps, but I think some schemes too - PLT Scheme perhaps?) also have optional Type Hinting that allow the program to run quickly too.

twp• 16 years ago

Compared to traditional static languages (C++, Java), Python offers an order of magnitude of improvement in productivity.

Compared to older scripting languages (Perl), Python offers a sane language and a rich programming environment.

Compared to other current popular scripting languages (Ruby), Python offers a solid runtime and better out-of-the-box performance at the expense of some elegance and fluency.

Comprare to some very interesting languages (Haskell, Erlang, Scala) Python is more widely available and allows you to produce good, portable, well-performing code quickly and easily.

It's a state of flux. Right now Python is my language of choice (after ten years of Ruby) and it touches that sweet spot of expressiveness and performance. However, the bolted-on features (verbose object system, clumsy metaprogramming) deliberate crippling of language (e.g. one line lambdas) and the underlying message of "there's one way to do it" (which is my way, because I know better than you) will, for me, mean that eventually I will move on. More expressive, coherent and flexible languages will eventually reach Python's current level of maturity and performance, and then it will be time to switch. It could be MacRuby, Scala or F# (but probably not Haskell or Erlang). Who knows.

But, for now, Python is a good choice.

Edit: typos.

foldr• 16 years ago

>Compared to other current popular scripting languages (Ruby), Python offers a solid runtime and better out-of-the-box performance at the expense of some elegance and fluency.

Although, Ruby doesn't have any iteration constructs as nice as Python's generators. (You could create them using call/cc, but I expect the performance would be awful).

scythe2• 16 years ago

May I suggest Lua with metalua? It seems to have most of what you're looking for (and luajit is very fast).

ThinkWriteMute• 16 years ago

Comprare to some very interesting languages (Haskell, Erlang, Scala) Python is more widely available and allows you to produce good, portable, well-performing code quickly and easily.

Yeah, absolutely not true. You need to do a little more research before you say very silly things like this.

jorgeortiz85• 16 years ago

Comprare to some very interesting languages (Haskell, Erlang, Scala) Python is more widely available and allows you to produce good, portable, well-performing code quickly and easily.

Scala code compiles to Java bytecode, and performs comparably to Java code. It's just as portable and performant as Java, which means very portable and fairly performant.

jacobolus• 16 years ago

Is there any evidence for this supposed trend? Or will these “many” programmers remain merely a vague hand-wave?

pgbovine2• 16 years ago

i've found that demanding empirical evidence from a blog post often leads to disappointment ;)

brehaut4• 16 years ago

Right on the money ;) All the evidence i have is purely subjective and anecdotal, but I personally know of a number for whom this is true.

imok20• 16 years ago

As additional anecdotal evidence, this is true for me, though I'm still using Python for my next project simply because it's a proven language with proven libraries.

But I'm certainly ready and interested in moving toward a more functional language -- I've always been interested in the Lisps, and have always tended to write code in the functional style, where possible (even though my first language was Java...)

joe_the_user• 16 years ago

I left Python for c++ because the job called for c++.

But there wasn't anything in Python that I found compelling so I never came back.

As a scripting language, Ruby seems so much more compelling, despite it's friendliness seeming to doom it to eternal slow bugginess. I'd go back to Ruby before Python.

But I'm more likely to add Python support to an app I'm developing, since I know it's more sensible.

Python's functional aspects seem chronically undeveloped so leaving it for a real functional language makes sense. But functional approaches seem limited in the "real world".

silentbicycle6• 16 years ago

The idea that people inevitably give up on their toolchain and move to another language en masse strikes me as a bit weird. I'm sure some people find itertools a good reason to learn other languages, but it's not some massive exodus. It's possible to use more than one language, you know. On the same project, even.

brehaut4• 16 years ago

Python is currently in a perfect storm for experienced users to want to move away. If it was just itertools it wouldn't be the case.

Point the first: Python 2.x to Python 3.x if you already have to make a transition, there is more impetus to make a transition to something that suits how you actually work.

Point the second: itertools is partly a catalyst, partly an indicator. Once you get fluent with it, it illuminates pain points.

Point the third: There is a new batch of functional languages that support the style that is a pain point, as well as fixing other issues that hurt in python.

Lastly, not all python programmers will want to make the shift. But the ones fluent with itertools are more likely to be the subset that will.

mattdw3• 16 years ago

I posted it because it's largely true of me. I'm still using Python (with Django) for my day-job, because I know the libs and the tools and the pitfalls, but almost all the code I've written for myself in the last year has been Clojure. I toyed with Haskell a little, but the type system killed me, and Scala wasn't quite functional enough (in the FP sense) for my tastes.

camccann• 16 years ago

From the "data is not the plural of anecdote department", I can say that personally, the more I use languages from the Lisp and ML families, the less desire I have to use Python, despite having spent a fair amount of time tinkering with it in the past. Alas, I currently use none of these at my day job, but any hobby projects I start in the future are very unlikely to be in Python.

dkersten2• 16 years ago

I am in EXACTLY the same position, though I will be leaving my day job in about a month to work on personal projects/contract work/startup ideas and I will be using a lot of Clojure for these projects. My main startup project is being constructed from a little C++ :-(, a lot of Clojure and some Yeti (ML derived JVM language).

dkersten2• 16 years ago

I have also observed this trend and commented on it here http://news.ycombinator.com/item?id=1087287 where other HN users have also confirmed it.

My own Python code, over time, became more and more functional in style until eventually I jumped ship to Clojure and Yeti (and over the past week, F#).

mbrubeck3• 16 years ago

I doubt anyone has valid stastics to answer this, but I'll throw in yet another anecdote. For about five years I used Python for all my personal projects, then for a couple years I switched to Ruby because I was using it at work. Last year I switched to Haskell for everything from simple command-line scripts[1] to larger programs[2] to silly recreational programming[3].

[1] http://github.com/mbrubeck/outline-grep

[2] http://limpet.net/mbrubeck/2009/10/30/compleat.html

[3] http://github.com/mbrubeck/mynock

LBarret2• 16 years ago

Just a data point, but I am in this position, for the reason listed in the blog post.

When I say in this position, I mean that I would like to switch but won't in the next future (too costly).

sfk• 16 years ago

Well, on blogs Haskell, F# and clojure are popular, but in the words of past decades:

Show us the code.

hga• 16 years ago

F# and Clojure are rather new and Haskell is only now maybe coming out of it's "Avoid success at all costs" phase (or so I judge by the Coders at Work interview of Simon Peyton-Jones).

jrockway4• 16 years ago

I've gotten paid to work on a lot more Haskell than Ruby or Python. The languages you see depends on the languages you want to see.

Haskell is really quite adequate for real work these days. It has the libraries you'd expect, and it's a nice language with a nice runtime.

It's also worth noting that blog-popular languages are blog-popular for a reason. The people that blog about programming are the ones that need tools that give them a bit more productivity between the occasional nap or meal; the people that use the more esoteric languages want to solve so many programming problems that a sharper tool is essential to their survival. (There is only finite time, after all, and programming languages are easier to build than time machines.)

In contrast, the "popular" languages are mostly used by people whose goal is to warm their chair for 8 hours and receive a paycheck. There are a lot more people that want money than that want creative time with a cold, unloving machine, so it follows that the languages where you can get paid and not care are going to be most popular.

But if you are the second type of person, there is little advantage in worrying about what is "popular" in "the real world". You don't want to be in the real world.

Paddy31182• 16 years ago

Python, popular!!! that is just so great! I've watched it grow for over a decade and a half. Python got were it is, largely without the big initial push of large companies.

Maybe people now blog about python because the community has always tried to be friendly. It is usually nice to blog about Python, and have fellow enthusiasts give measured, polite feedback.

"... so it follows that the languages where you can get paid and not care are going to be most popular."

But it does not follow that there are less caring, articulate, and productive souls in a more popular languages camp.

- Paddy.

andrewljohnson• 16 years ago

False: "There is an observable trend in Python programmers that results in a reasonable section of them moving to functional programming languages."

True: There is an observable trend that more and more people are using Python, and there is a smaller set of people that is not a subset of Python in any way that is toying with functional programming, usually for academic or masturbatory purposes.

bobbyi• 16 years ago

The reason Python is such a joy to use is that it hasn't felt the need to be all things to all people.

Those of us who use python are often coming from C++, perl, etc. and know all too well what happens to languages that try to add every feature that someone might find useful.

If you want to use a lazy purely functional language, you know where to find one. It's not python.

If you want a language that allows you to express your inner emotions by using your personal pet programming style, this isn't it either.

nostrademons3• 16 years ago

"If you want a language that allows you to express your inner emotions by using your personal pet programming style, this isn't it either."

That would be Lisp. ;-)

moe2• 16 years ago

Oh really? I would have said perl or perl-reloaded (aka ruby)? ;-)

jchonphoenix3• 16 years ago

As a programmer who has used variants of SML (F# is Microsofts variant), Haskell, and Python, I would say that in terms of concise expressiveness, you have a point that the functional languages match Python's. However, the true power of programming in Python is that the programming logic is easy to understand. If you are actually experienced in functional programming, you'll realize that it is, in all definitions, code. Python is more like english and thus, the appeal will always be there.

mbrubeck3• 16 years ago

"Python is more like english"

Having written non-trivial programs in Haskell, ML, and Python, I agree with this, but I would add: Functional programming is more like mathematics.

English expresses some ideas more clearly, while mathematics is better for others. And notations that are more "mathematical" often have other nice properties like composability and equational reasoning.*

*The best example of what I mean by equational reasoning is the Sudoku functional pearl: http://www.cs.tufts.edu/~nr/comp150fp/archive/richard-bird/s...

jrockway4• 16 years ago

The paper you posted is a good example. The conclusion is:

    sudoku = map (map head) . search . prune . choice

Note how the "code" never mentions the data to which it is applied, namely the Board object. This is why I can't agree with the GP's argument that FP is closer to "code" than anything else -- I think it's as far from "code" as you can get. There are no "instructions" here, just an equation representing something complex ("the solution to sudoku"), in terms of slightly simpler parts. The simpler parts are, earlier in the paper, the complex parts; and are of course described there by simpler parts.

And, this is not just something that's useful for esoteric mathematics papers; my "enterprise" code in Haskell looks very similar.

jchonphoenix3• 16 years ago

"Functional Programming is more like mathematics." I would completely agree. In fact, I would add to that and say FP is mathematics because its "Functionally" based and was originally created for easy inductive proofs.

However, from a readability standpoint, english is better than mathematics, and readability counts for quite a bit if you aren't planning on throwing away your code.

jrockway4• 16 years ago

If you are actually experienced in functional programming, you'll realize that it is, in all definitions, code. Python is more like english and thus, the appeal will always be there.

I disagree. Functional programming is about as far away from the machine as you get. When I write functional code, I feel like I am describing the problem I want to solve in great detail, just as I would if I were writing about it. (And despite being "far away from the machine", the code still runs quickly. There is something to be said for describing a solution to your problem instead of telling the CPU which bits of main memory to flip.)

As you go down the spectrum from FP to OOP, your code becomes more code-like. Create object. Call method. Save return value. Pass value to another object. Get result. This is still pretty abstract, though, and a fine way to code.

Then you get to procedural programming (C), and then your code truly is code. Allocate memory. Move bits from register to memory. Set interrupt. It's hard to even see an algorithm in there, there's just so much code.

In functional programming, things are often so abstract that you define functions in terms of other functions without even mentioning the data you are processing. Let's face it; that's not "code".

silentbicycle6• 16 years ago

"Functional programming is like describing your problem to a mathematician. Imperative programming is like giving instructions to an idiot." -- arcus, #scheme on Freenode

(from the Scheme quotes file at http://community.schemewiki.org/?scheme-fortune-cookies)

jchonphoenix3• 16 years ago

I guess I should be more specific. When I say code, I mean "cryptic and unintuitive." Yes, you may say FP is intuitive if you've used it for awhile, but at a glance, Python is much friendlier than ML.

justin_vanw5• 16 years ago

Functional programming simply isn't any better than the usual style of python development.

I have used much of itertools. When you need to do some complex iteration through lists, itertools is great. When you need to pull an address out of a web page, itertools does absolutely nothing to make that easier, nor does any other functional programming technique.

So here is the difference. Procedural style is better for dealing with a problem. Functional style is great for applying a solution to lots of pieces of data in a structure that can be iterated over or recursed into. The hard part is dealing with each problem, not the part where you apply it.

mattdw3• 16 years ago

> When you need to pull an address out of a web page, itertools does absolutely nothing to make that easier, nor does any other functional programming technique.

I'd say an approach like "iterating through all nodes, filtering for the one(s) you want" is quite functional. Add lazy iteration to that, so you're only parsing/consuming as much of the document as necessary, and on-demand, and you've got a really nice, _functional_ solution.

[edit: toned it down a little.]

justin_vanw5• 16 years ago

Ok, that is great so long as you are processing XML. Actually, it isn't great even if you are processing xml.

What is the functional way to get the 3rd - 8th li in the 2nd ul where the ul has id="foo" and is in a div (either directly or as a decedent) where the div is directly before the 'comment' <-- content area --> ?

You use XPath, or CSS after finding the comment with a regex. People get incredibly retarded about functional languages, but in reality the only software that I have used that is written in one is XMonad. So in your mind everyone who actually gets things done is using the 'wrong language'? Or an inferior one?

Name one big success story of functional programming. There isn't one. Nobody uses it who actually cares more about getting things done than reading self congratulatory blogs between CS classes.

[in no way toned down]

yesimahuman• 16 years ago

The only reason I'm starting to leave Python is because of performance. When you see a program that used to take 2 hours to run in python now taking 10 minutes in Java, it kind of makes you realize you've been making a mistake by using it places it doesn't belong.

viraptor• 16 years ago

There's always Jython for occasions like this one...

nostrademons3• 16 years ago

Jython is closer to Python's speed than Java's.

Or if you mean Jython with the speed-critical bits written in Java...well, there's always C for that.

LBarret2• 16 years ago

Did you try cython/numpy/fileinput ?

just curious. Our team met perf problem in the past and each time, a rewritten code or even a rewritten module was all it took to get back on track.

gte910h• 16 years ago

Try:

Weave Cython SIP

Python is really easy to tie into other languages for the slow parts.

dschobel• 16 years ago

I've had the opposite experience with C#. MS' adding of lambdas and deferred list operations in 3.5 has delayed my switching to F# for all of my heavy number crunching work (For those who have yet to try ruby, python, scala, c# et al-- being able to switch between imperative and functional style on a per task basis is a wonderful way to work).

In the real world, suffering a few weeks of diminished output due to learning new syntax, idioms and best practices is not always (dare I say, rarely) feasible so making an existing language more powerful through the addition of functional constructs can only extend their productive life.

apower• 16 years ago

Is this like the exodus of the Ruby programmers? Remember Ruby was on rage couple years ago? But suddenly became out of favorite due to various issues. Looks like Python is the same way. People play around with it and find out all kinds of issues. When a new language appears (Clojure), people leave in doves. Just another fad.

brehaut4• 16 years ago

I've been using python for about 8 years before feeling like i wanted to move. I think that puts me well outside the fad window.

micheles• 16 years ago

I am a Python programmer. I have toyed with functional languages (Scheme, SML) for many years. Over the time my code is becoming more and more functional, in every language I code in. Nevertheless, I do not plan to quit Python at all. Readability counts in my book and I do not see Clojure to be more readable than Python. Moreover, I am not considering F# since it is platform specific and I do not develop on Windows. In my experience Python is not losing programmers; perhaps 0.1% of them are moving to functional languages, but more are coming from C++, Java and even Ruby.

manpreets7• 16 years ago

Python holds a strategic middle ground. It's far more productive, expressive and concise than the traditional procedural and OO languages. Although not as expressive and elegant as the new FP languages like Clojure and F#, the latter lack massive libraries and modules that have been written for Python. That makes them useless for any 'real' work.

Python is seeing some heavy use by the scientific community due to tools like NumPy, matplotlib, libsvm and what not. Good luck finding that range of independent libraries for a new FP language. I am not dropping Python in the near future, but I am watching.

mattdw3• 16 years ago

Clojure can use (with very little mismatch) any available Java/JVM libraries.

F# can use anything available for the CLR/.NET.

Libraries are not really an issue.

petsos• 16 years ago

Judging from the comments, people leave Python to do toy projects in functional languages.

My guess is that after using functional languages in real projects, most of them will go back to Python.

Paddy31182• 16 years ago

I am an active contributor to www.RosettaCode.org where I browse many languages. What I have seen of the functional languages doesn't make me want to switch.

Despite knowing Pythons functional tools, it wasn't Python that introduced me to that style. I find Pythons mix comfortable and happily switch between programming in Python 3 and Python 2. Maybe the OP hopes to start a trend?

- Paddy.

"On iteration": Why people leave Python

Related Stories