The Zen of Python by Example

http://artifex.org/~hblanks/talks/2011/pep20_by_example.py.txt

by pjo • 15 years ago

199 63 15 years ago

ff0066mote• 15 years ago

Forgive me if I am wrong, but I believe that section 1 contains mistakes. In the implementations of evens_only, shouldn't both of the places where modulus is used be preceded by "not"?

This isn't meant to detract from the truth that "Beautiful [code] is better than ugly," I merely thought the implementation should be correct.

rodion_89• 15 years ago

or better yet i % 2 == 0

l0nwlf3• 15 years ago

I would go for more pythonic "not i % 2".

highfreq• 15 years ago

I agree. Using the truthiness of non-boolean values has always seemed questionable in any language (Javascript is possibly the worst offender). For many statically typed languages this is caught as a syntax error, which is nice for those of use prone to occasionally typos.

rikthevik• 15 years ago

Yeah, I don't think you can use i % 2 as an equivalent of i % 2 == 0. Maybe it's a python 3 thing, but on my 2.6.1 the mod operator just returns the remainder, it doesn't change its output if it's a boolean context (which would be perlish and highly unpythonic).

I think i % 2 is just straight up incorrect.

If we're talking about style, I think i % 2 == 0 is much more clear than (not i % 2). It says explicitly that the remainder of i divided by 2 is 0.

hellrich• 15 years ago

It seems to work in older Python versions (tested with 2.5). I think the new implementation is more natural: even%2 = 0, and 0 is false, while uneven%2 = 1, and 1 is true.

coolgeek2• 15 years ago

Hey, I was at the RedSnake meetup where this was presented on Tuesday night!

Go Hunter!

phillytom• 15 years ago

It was a great event - there is a short writeup of the other talks here: http://trevmex.com/post/3198570101/redsnake-1st-annual-phill...

dorkitude• 15 years ago

I love this document. This stuff is my porn.

That said, I'm having quite a hard time grappling with section 4.

First, what is the 'users' keyword all about in the "bad" area?

Second, why is ORM the "bad" approach, and manual construction of SQL strings the "good" one? It seems to me that the abstraction and centralization of query language is paramount (examples: to afford oneself an easy one-day migration to GQL/BigTable, or to a document-oriented database like MongoDB or the upcoming Couchbase).

Legion• 15 years ago

Just like how he reached for PyUnit for "readability counts", I definitely think of ORMs over raw SQL when it comes to "readability counts". Especially when you're actually working with data, not just the table creation step.

endtime2• 15 years ago

Agreed completely. The difference between a good ORM and raw SQL is like the difference between, well, Python and assembly, IMO. Perhaps the difference is a little less extreme, but it's similar in spirit.

mcginleyr1• 15 years ago

The pretty examples aren't in order. In this case the doctests are the "pretty".

rhoeft• 15 years ago

I think maybe the ORM is being put forth as the good approach. In the 'simple is better than complex' section using the ORM was the 'complex' solution and using json was the 'simple' solution. In 'complex is better than complicated' the ORM is still the 'complex' solution, but that wins over the 'complicated' manual SQL solution.

l0nwlf3• 15 years ago

Here is the history of "Zen of Python", aka ">>>import this" :

http://www.wefearchange.org/2010/06/import-this-and-zen-of-p... , a blog post written by Barry warsaw.

On a side note, other eggs are :

>>> from __future__ import braces

>>> import antigravity

nmb• 15 years ago

For the evens_only implementation, is there a reason to use map/filter vs a generator? eg:

   evens_only = lambda nums: [n/2 for n in nums if n%2==0]

seems to work just as well.

l0nwlf3• 15 years ago

I think that is List Comprehension and not Generator. 'Return' would be replaced by 'Yield' in case of a generator.

drdaeman• 15 years ago

With [...] you are doing the same as list(...), so your example would return a list:

    >>> evens_only([1,2,3,4,5])
    [1, 2]

If you want a generator instead:

    evens_only = lambda nums: (n/2 for n in nums if n%2==0)

So you'll get:

    >>> evens_only([1,2,3,4,5])
    <generator object <genexpr> at 0x7f2ef79eaa00>

I don't know which one would be more Pythonic, clean and simple lambda one-liner or two-lined simple function.

KirinDave4• 15 years ago

I couldn't help but feel irritated by this document. It's cute and all, but it implies Python is somehow unique—culturally or technologically—in grasping at these concepts. You could easily have written this as a Ruby or Scheme manifesto with nearly identical wording and had a valid message.

And then, in our very first example, there is a use of lambda, a misnomer of a concept Python doesn't really support? My brow furrowed slightly more.

I can't help but feel like these people are celebrating their investment in their tools rather than the virtues of the tools themselves.

rmc• 15 years ago

You could easily have written this as a Ruby or Scheme manifesto

Yes, but this practically is the Python manifesto.

KirinDave4• 15 years ago

I'm saying it's also the Scheme manifesto.

TheBoff• 15 years ago

The first section is pointing out list comprehensions as a better syntax than lambda, I believe. Also, the zen of python has been around for a long time: http://www.python.org/dev/peps/pep-0020/.

There's an import statement that will print it I believe, but I'm not sure exactly what it is.

ronnix2• 15 years ago

$ python -c "import this"

KirinDave4• 15 years ago

It's a better syntax than Python's 10-year-out-of-date notion of lambda. For example, this is what a more modern language does with lambdas:

    numbers.filter( _ % 2 == 0 )

This document is old, so we can probably excuse it... but how many Python folks know that?

baddox3• 15 years ago

Awesome. Here it is in a paste bin with syntax highlighting:

http://paste.pocoo.org/show/336126/

baddox3• 15 years ago

This is a bit off topic, but what are your opinions on the validity of the 80 character width restriction? I think the 80 character limitation is increasingly rarer to actually encounter these days. Even people developing via a shell will almost certainly have much more than 80 characters of width to play with.

We've finally accepted that web developers can expect visitors to have a higher resolution than VGA, and those are potentially non-tech-savvy users. Can't we expect developers to have more than 80 character width editors? Plus, I think splitting lines is pretty ugly.

ronnix2• 15 years ago

I think 80 characters is getting a bit narrow, but some people like to be able to open several files (e.g. the code and the tests) in VIM or Emacs, with the buffers vertically arranged next to each other.

j_baker2• 15 years ago

On the other hand, I prefer not to have all of my desktop taken up by my editor. Usually, I dedicate half of it to my editor and half to a command-line. Now, 80 characters might be a bit too small, but I think it's a good goal to shoot for as long as you bear in mind that it's a guideline and not a hard rule.

jamesgeck0• 15 years ago

On my 5:4 aspect monitor, 80 characters is just wide enough so that I can fit two editor columns side by side at a comfortable font size without having to scroll horizontally. I suspect that if I had a widescreen monitor, I could shrink the font size slightly and see three columns of 80 characters each. But that would just be extravagant.

(Of course I have a second monitor, but that usually has documentation and a terminal and IRC on it.)

glenjamin• 15 years ago

The idea of restricting your width to [a number] makes sense for the tiling reasons mentioned by the other commenters.

However, since everyone is shipping widescreen monitors (which I think means UI design should stop making tall toolbars, but I digress), I find that a width of 100 characters suits my code better.

eru4• 15 years ago

I'm using my monitors rotated 90 degrees (i.e. portrait instead of landscape). Unfortunately, there's only 1060 pixels from left to right.

Also, the restriction to something around 80 characters stemmed from technical considerations---but it may be good for readability, too. After all 66 characters per line is the rule of thumb for type setting books. Newspapers also divide their pages into narrow columns.

Macha• 15 years ago

I used to be dead set against 80 character limits, and considered them a historical relic. But nowadays I'm actually in favour of them. On my 1440x900 screen, 80 characters allows me to have two files side by side in vim, or vim and a terminal side by side. Likewise, I'd imagine a programmer using an editor or IDE that's heavy on screen space would probably have about 80 characters of space for the file in a similar resolution.

Finally, in some cases, it's just more readable. Take the following code:

    some_list = [a_long_variable, with_contrived_names, for_an_example, insert_your, own_parallel, if_it_bothers_you]

versus

    some_list = [
        a_long_variable,
        with_contrived_names,
        for_an_example,
        insert_your,
        own_parallel,
        if_it_bothers_you
    ]

RiderOfGiraffes• 15 years ago

I really, really hate lines that stretch the entire width of a window - it makes it hard to find the right line on the fly-back. I've reached the end of the line, I'm looking for the start of the next line, my eyes flick back to the left side of the screen. Now, where was I?

When text stretches to take up the entire width I don't use full-screen windows. I adjust them to have about 10 words per line, and I can scan down really fast. Similarly for programming. Don't make me scroll through a 6000 line function, and don't make me read along a 6000 character line.

Yes, 6000 characters may be an exaggeration, but where is the limit? Where is the point where the line is simply "too long". Answer? It's where readability drops off, and for me, that's about 10 words, or about 60 characters.

singingwolfboy• 15 years ago

A quarrel -- Pygments is written in Python, and is appropriately modular. Why did the author use subprocess and create new Python interpreter to run Pygments, when he could have just imported it and run it directly, without kicking off a new interpreter?

d0mine3• 15 years ago

There is a bug in #1. The condition should be `i % 2 == 0` to get even numbers:

  def halves_of_evens(nums):
      for i in nums:
          div, mod = divmod(i, 2) #NOTE: it might be slower
          if mod == 0: #NOTE: `not mod` is not explicit enough
             yield div

#2: Function level import is a bad practice though it might be justified in this case.

#3: `users` is not defined. `sqlalchemy` names are not defined. The file should be opened for writing in the json example.

#5 It is simpler to allow the animal to know its kind:

  def identify(animal):
      return animal.__class__.__name__

or just:

    animal.kind

#6 `elem.xpath('./a/@href')` is simpler than

  [a.attrib[’href’]
   for a in elem.find(’./a’)
   if ’href’ in a.attrib]

Also `elem.iterlinks()` could be used http://codespeak.net/lxml/lxmlhtml.html#working-with-links

There is no `lxml.CSSSelector` (python-lxml 2.2.6-1). Use `elem.cssselect`

#7 You don't need to add `ifmain` stub in every module:

  $ python -mdoctest module_with_doctests.py
  $ python -munittest module_with_unittests.py

# 8 & 9: The first variant of `make_counter()` won't work without `nonlocal i` -- it leads to `UnboundLocalError` otherwise.

Despite the fact that it works on my machine on all available Python implementations such code should be avoided:

  assert float(’0.20000000000000007’) == 1.1 - 0.9 #XXX horror!

# 10 & 11: Use stdlib's version:

  try: import json
  except ImportError:
       import simplejson as json

It is a matter of preference but it makes testing more predictable.

#13 `__builtins__` is not defined on jython:

  try: import __builtin__ as builtins
  except ImportError:
       import builtins # py3k

pyre2• 15 years ago

  > #2: Function level import is a bad practice though
  > it might be justified in this case.

Ah, but:

  > Special cases aren't special enough to break the rules.

;-)

siika2000• 15 years ago

Why is function level import considered bad practice?

d0mine3• 15 years ago

Deadlocking on the import lock might be introduced by an innocent code that uses a library with function level imports.

In general globals should be written at a global level.

dustingetz• 15 years ago

http://stackoverflow.com/questions/1699108/local-import-stat...

j_baker2• 15 years ago

You know, the Zen of Python makes a for a really cute thing to read, but even Guido himself has admitted that it requires quite a bit of interpretation and doesn't apply in all cases. Generally, I feel that applying it directly to code is tricky (except for trivial pieces of code like these). I think it's better to treat the Zen of Python as a broad vision for how Python code should look rather than as a concrete set of rules for code.

For example, explicit isn't always better than implicit. I mean, by that standard alone, the idea of a garbage collector going around implicitly freeing memory would be terrible.

pyre2• 15 years ago

I think that the issue is that some of the statements can come into conflict. You can easily make something that is more complex (i.e. less simple) in an attempt to make something that is more explicit and less implicit.

hblanks2• 15 years ago

By way of an afterword, I'm both pleased and surprised with the thought and attention people have given to these slides, which I made for a perhaps too hastily prepared, 10 minute talk at this Tuesday's joint PhillyPUG / philly.rb meeting. (Many thanks to the organizers, other speakers, and attendees -- it was a great turnout and a good time, too.)

I agree with several commenters that, like any collection of aphorisms, the lines in PEP 20 are not real rules but rather rules of thumb. Nevertheless, rules of thumb, or heuristics, are surprisingly important to the practice of engineering,* and perhaps even more so to the work of writing software. As such, nothing in PEP 20 should be followed hard and fast, but almost all of it is worth considering. Also as such, there's nothing (except for the comment on Guido) that makes PEP 20 relevant only to Python. It happens that these rules have a particular currency in the Python community (i.e., when people talk about being Pythonic, they're often talking about something described in the Zen of Python). But if you work in a different language, there's probably something worthwhile in it there, too.

On a different note, it is flattering to find this code, hacked out such as it was, has been reviewed by a far larger number of peers than the (comparably) few, albeit exceptional, engineers I work with at Monetate. Although it was hardly intended to be read on its own, or to run beyond the feeble purpose of generating its own slides, I have tried to correct what errors people have found. Ambiguities, of course, remain, but you can find an updated document at the original URL, or a diff of the changes at:

    http://artifex.org/~hblanks/talks/2011/pep20_by_example_2011_02_10_edits.patch

To respond to a few specific or general points:

- Yes, of course, importing from within in a function is almost always a bad idea (the only exception I've ever seen is Google App Engine, and that's obviously just a performance hack). Nor would you ever want to actually convert your floats to strings and back -- my only point there was to illustrate the fact that yes, those are the rules of floating point, and so you can't expect Python to break them for repr().

- The order of examples is not necessarily bad first, good second. (Alas for inconsistency!) So, for instance, ORMs may be complex, but they're often much better than writing your own SQL, which even in this trivial example starts to get complicated. Similarly, doctests often make code more readable than unit tests hiding in some other file -- although in honesty the doctest/unittest split is probably a false dichotomy; for some things, doctest is much cleaner, for others (like things that require setting up a DB schema first), unit tests are.

- Finally, I still didn't have the wherewithal to call pygments directly (it's certainly easy and better; I just knew the command already). But I did add HTML output, since this the web, after all, to:

    http://artifex.org/~hblanks/talks/2011/pep20_by_example.html

* Cf. Billy Koen's enjoyable discussion of engineering heuristics, Discussion of the Method or Discussion of the Engineering Method. (Some chronicle is available at http://www.me.utexas.edu/~koen/OUP/HeuristicHistory.html)

Locke16892• 15 years ago

Do you by any chance know Jack Lange? That's the only other time I've ever seen an artifex.org domain.

hblanks2• 15 years ago

Yes, I do. Jack's an old friend an another one of the admins for Artifex.org.

suraj• 15 years ago

I do not like example 4 "Complex is better than complicated". Use of string concatenation is going to lead to SQL injection bugs and SQLAlchemy version actually looks better to me.

Can somebody please enlighten me on how using raw sql is preferable?

Luyt• 15 years ago

You could use dbapi's query parametrisation instead of string concatenation, or maybe use sqlalchemy's SQL Expression if you don't want to write SQL.

ORMs are not a panacea. Especially when the database schema gets more complicated they require more setup and tuning, and there's always the problem of how much data the ORM should suck in vs. how lazy it can behave.

ludwigvan• 15 years ago

PDF version is easier to follow:

http://artifex.org/~hblanks/talks/2011/pep20_by_example.pdf

sausagefeet2• 15 years ago

I have a hard time agreeing with this document. In many cases it is hard to tell which part of the wrong solution they are actually correcting. For example on the one where they write measurements out to a file, in the first case they use sqlalchemy to write to a sqlite db and in the 'correct' case they write json to a file. So what is the issue that is be corrected here? Is using sqlite without sqlalchemy ok? Both?

drstrangevibes• 15 years ago

I find all this discussion about the rules slightly amusing, as it is Zen. The rules are there to instruct the apprentice, to guide adept, but there are no rules for a master. shu ha ri.

http://en.wikipedia.org/wiki/Shuhari

I do not claim to be a master

The Zen of Python by Example

Related Stories