Python and the Principle of Least Astonishment

http://lucumr.pocoo.org/2011/7/9/python-and-pola/

by mattyb • 15 years ago

207 49 15 years ago

lucumr.pocoo.org

Kilimanjaro3• 15 years ago

Differentiating properties and methods from internal methods is as simple as using another char, like dots and colons. So box.length would not be the same as box:length, that way you could assign all names you want without fear of collision. Poor choices in language design will always haunt you til the end of times.

That being said, python is my favorite language right now and the only complain I have is about underscores which I try to avoid.

Now, GO being a new language, I can't really understand why it does not implement methods for primitive types like "hello".ToUpper(), instead we have to import "strings" and call strings.ToUpper("hello").

Easier for the compiler but harder for the programmer has never been my mantra.

lucian1900• 15 years ago

I'm not sure about different namespaces for special methods. Python's object system is so self-referential and so transparent that it'd be surprising.

Go devs are of the opinion that what behaves like a function should be a function, with no exception. It's also the reason they don't do operator overloading, which I think is just dumb.

dalke2• 15 years ago

Would you mean this to be a namespace which only the Python implementation could access? For example, if the protocol "it:next" is added (which is equivalent to next(it)) then is there any way to support that syntax in an older version? With next() as a builtin it's easy; try: next except NameError, and in case of exception, implement the function yourself. But if I can't implement it myself then there's no good migration path.

With your particular choice of ":", then what would "d={a:length()}" be? Currently it's a 1-element set. What would "words[middle:length()]" do?

Kilimanjaro3• 15 years ago

"is there any way to support that syntax in an older version"

Nop, that is exactly what I meant with decisions haunting you forever. You better start a new language than radically change the current implementation.

With regard to my particular choice of ':' there are three solutions:

1. pick any of the 100k unicode chars for internal methods but imperative is to differentiate them to freely use attributes and methods as pleased.

2. force the compiler and the coder to disambiguate the possible same use as internal method: first assign then use n=a:length(), d={n}

3. design the language to use .. as slicer and {a=b} as dictionary assignment.

Again, we are talking about design decisions for a new language, not to change the core of an existing language which is not an easy task. That is exactly why language design is not an exact science an very few make it to stardom.

njharman2• 15 years ago

how is ':' less onerous than underscores i.e. '.__'?

Sounds like you just have irrational hatred of underscore (or possibly love of colon?)

btw having two or more access operators '.', ':' and distinguishing between "internal/language" and "user" methods is a poor language design choice over having a standard but unenforced __ means "special" and "internal/language" and "user" all being coequal in semantics, syntax, rights and responsibilities.

Kilimanjaro3• 15 years ago

"how is ':' less onerous than underscores i.e. '.__'?"

There is no difference at all, just one char vs five if we count ending double-underscores too, and more visual noise. They both accomplish the same.

Being both the same, what difference do you think there is between, in one hand .init() and .__init__(), and on the other hand .init() and :init()

Just syntactic sugar, I rather pick my colon ;-)

anonymoushn2• 15 years ago

For extra fun....

  a = 5
  def print_a():
      print a
  # Prints 5
  print_a()
  
  def print_and_assign_a():
      print a
      a = 2
  # raises UnboundLocalError
  print_and_assign_a()
  
  class PrintOnInitSet(set):
      def __init__(self, *args, **kwargs):
          print "init!"
          set.__init__(self, *args, **kwargs)
  # Creates a PrintOnInitSet and prints "init!"
  a = PrintOnInitSet([1,2])
  # Creates a PrintOnInitSet and prints "init!"
  b = PrintOnInitSet([3,4])
  # Creates a PrintOnInitSet and prints nothing
  c = a | b

kbd3• 15 years ago

Well, your first example makes sense. It ensures you're always referring to the same-scoped 'a' throughout your function. FWIW if you said 'global a' at the start of 'print_and_assign_a', Python wouldn't have a problem:

    >>> a = 5
    >>> def print_and_assign_a():
    ...     global a
    ...     print(a)
    ...     a = 2
    ... 
    >>> print_and_assign_a()
    5
    >>> print_and_assign_a()
    2

Your second example, however, seems to show an implementation detail of Python leaking out, and is probably a bug. Union starts off by making a copy of 'a' and adds the elements of 'b' to it, rather than creating a new object and adding the elements of both.

On a hunch that PyPy's implementation would be less special-cased, I installed PyPy and and it actually does print "Init!" on "c = a | b". So, I vote bug in CPython rather than an inconsistency in the language.

One more question arises, however. Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"? I overloaded 'add' on your PrintOnInitSet and it's not:

    >>>> a = PrintOnInitSet(); a.add(1); a.add(2)
    init!
    Called add with item: 1
    Called add with item: 2
    >>>> a = PrintOnInitSet([1,2])
    init!

anonymoushn2• 15 years ago

The 'print_and_assign_a' case comes up more often with a in an enclosing local scope, so you couldn't really get it to work prior to the introduction of 'nonlocal' in Python 3. The best you could do was something silly like

  a = [5]
  def print_and_assign_a():
      print(a[0])
      a[0]=2

It's cool that the second case works in other implementations. I hadn't thought to test that.

ak217• 15 years ago

> Should "a = set([1,2])" be equivalent to "a = set(); a.add(1); a.add(2)"?

Why should it be? Adding elements one-by-one is slow and shouldn't be forced on the language model level.

gasull• 15 years ago

I miss something like the Principle of Least Astonishment for design.

Why every time there's a revamp of a website the new design is more astonishing and less clear? Why the search button in Google Calendar is now blue?

Why new design is usually a tease on our muscle memory?

spacemanaki3• 15 years ago

This is a nice article, as a non-Python programmer who's dabbled a bit in order to read some random Python code, it gave me some appreciation for the "Pythonic"-way.

I have to say though, that the thing that astonished me the most about Python is the Java-like "closures". I sort of thought it would be like Ruby, which I thought was closer to Scheme and JavaScript, and then I realized that Ruby isn't quite like them either. (I know Ruby is close with its lambdas and blocks, but it's not intuitive to me.)

This isn't necessarily to the detriment of Python (or Ruby) but just something I observed when trying to explain Java's lack of closures to someone who only knows Python.

JeremyBanks• 15 years ago

I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's?

I haven't used Java a lot either, but from what I have done Python's function seems more similar to JavaScript than to Java.

irahul4• 15 years ago

> I have only a trivial amount of experience experience with Scheme, could you explain how its closures are different from Python's?

In pre Python 3, the closed over value isn't mutable unless it's a reference to a mutable object.

        def counter(num):
            def foo():
                num += 1
                return num
            return foo

    c = counter(5)
    c()

This won't work because you can't mutate the closed variable `start`.

This would work in languages with proper closures(Ruby, Perl, Scheme...).

Here is how you do it in Ruby:

    def counter(n)
        lambda { n += 1 }
    end
    c = counter(5)
    c[] # returns 6
    c.call() # Alternate syntax. returns 7
    c[] # returns 8

The above python will work in Python 3 if the closed variable is declared `nonlocal`.

    def counter(num):
       def foo():
           nonlocal num
           num += 1
           return num
       return foo

Or you can have workarounds in pre Python3.

    def counter(num):
       def foo():
           foo.num += 1
           return foo.num
       foo.num = num
       return foo

kingkilr• 15 years ago

Python has Java like closures? No comprende.

irahul4• 15 years ago

Java's closures are a pain as they need inner classes to emulate closures - Python's are a lot easier, hands down.

Java needs the local variables to be final for it to close over them. Python doesn't but you can't directly mutate it. Won't you agree in that sense the statement holds some truth?

irahul4• 15 years ago

> then I realized that Ruby isn't quite like them either.

I am curious. How does Ruby's closure differ from Scheme's?

spacemanaki3• 15 years ago

To be honest, I'm not really sure. I think it's a superficial difference. When I put this into a Ruby 1.9.2 REPL

    def foo (n)
        lambda {|i| n += i } end

I expected to be able to do this:

    acc = foo(0)
    acc(1)

But instead have to do

    acc.call(1)

Maybe there's another way...

Edit: oops you pointed out this example in another comment, sorry! Is there no way for the returned lambda to "feel" like a regular function?

nikcub• 15 years ago

you should read Armin's other posts about Python and webdev, if you haven't already:

http://lucumr.pocoo.org/tags/python/

rmccue• 15 years ago

Personally, I love that joining is ' '.join(list), since PHP (e.g.) works the same way, in that the first argument to implode() is the string.

xtal• 15 years ago

It being consistent with PHP is pretty much the same thing as being inconsistent.

andrewflnr2• 15 years ago

Overall it's a good article, but I don't fully understand his complaint about decorators, and I think he may not understand them. @foo and @foo() intentionally mean very different things. I don't see how you could "add a parameter to a previously parameter-less decorator" without drastically changing the meaning of the whole thing.

They may be tricky to wrap your head around, but there's nothing too surprising about decorators.

irahul4• 15 years ago

> I think he may not understand them.

I won't think so. He codes a lot, in general, and in Python. I have read some of his posts and code, and he has a deep understanding of Python and programming in general.

His complain was about @foo and @foo() requires different decorator implementations. If @foo by default meant @foo(), that makes introducing parameters at a later time a bit more straightforward.

I am not arguing about it being a valid expectation. I am just explaining what I think he meant.

MostAwesomeDude2• 15 years ago

If I have the following code:

    @auth_required
    def some_view():
        pass

And later I decide to change that decorator to be parameterized:

    @auth_required("basic")
    def some_view():
        pass

I have now created a massive amount of pain for myself if I decide to define auth_required() such that it defaults its argument to "basic", unless I force people to call it as @auth_required(), because of the differing callable signatures.

Some people feel @auth_required() is ugly. That's all.

the_mitsuhiko2• 15 years ago

> Some people feel @auth_required() is ugly. That's all.

The biggest issue here is backwards compatibility. Nowadays I just make a separate decorator that accepts arguments and keep the old one around unchanged. I learned my lesson :)

andrewflnr2• 15 years ago

Apparently there's something pretty big here I'm not getting. If you add a parameter to auth_required, of course you're causing a huge amount of pain because you're totally changing what auth_required means. It's going from being directly applied to some_view to generating a function that is applied to some_view. If every decorator was automatically called, every decorator would have to return a function. Among other consequences, you wouldn't be able to use property as a decorator.

Even if it's inconvenient, python's behavior here is still not surprising. It's perfectly consistent, and more general than automatically adding parentheses.

pak• 15 years ago

It's a bit dated, but the Python Cookbook does a great job of teaching you practical examples of Python if you already know other languages and want to quickly grok what is "pythonic." It starts with string crunching and hits just about every other general sysadmin use case, covering most of the standard library along the way.

hello_moto• 15 years ago

May I know why it is a bit dated? because I'm thinking to buy the book and the year is 2005, which is good enough for me since 2005 means Python 2.5 at least (or even more).

sateesh• 15 years ago

The 2nd edition of Python Cookbook doesn't cover Python 2.5. It covers language features that were there till Python 2.4. (http://oreilly.com/catalog/9780596007973) The 3rd edition of Python Cookbook is expected to appear by end of this year. http://dabeaz.blogspot.com/2010/12/oreilly-python-cookbook-p...

astrofinch• 15 years ago

"I was quite fond of Raymond Hettinger's talk about what makes Python unique because he showed a bunch of small examples that almost exclusively used the good parts of the standard library and hardly any user written logic."

Is this talk available anywhere?

room606• 15 years ago

I'm not certain but I think it was one of these

http://ep2011.europython.eu/conference/speakers/raymond-hett...

mfieldhouse• 15 years ago

Maybe this http://ep2011.europython.eu/conference/talks/what-makes-pyth...

bluesnowmonkey• 15 years ago

So... Python has len() because it would be too hard to standardize on .len(). Any by the way, len() calls .__len__(), on which we have standardized.

Got it.

aston• 15 years ago

The better explanation is that there are a series of fallback actions for all of the top level functions that delegate to the .__whatever__ methods.

For example, if you write a class that quacks like a collection that has a length (it implements .__len__) Python can use that to get a value for bool() even if you didn't implement .__nonzero__.

julian372• 15 years ago

When people inquire about the len() function it is mainly because it is (or at least looks like) a global function rather than an object method. In short, people wonder why you have to call len(x) instead of x.len(). This is a valid concern seeing that Python's standard library is, by and large, object oriented (c.f. string methods like "upper" or list methods like "append" or file methods like "read").

Neither the article nor your comment has a good explanation of why that is so. The article only harps on about why it's good that it's not called length() or getLength() or size() or getSize(), implying that len() is somehow the god-given name for this feature. And the fallback mechanism you refer to could just as well be implemented if len() was a method of the root class (object) rather than a global method.

I like Python a lot but this is one that I never quite understood and I believe there isn't really a good reason for it, I think it was just a historical accident. Might as well admit it rather than trying to come up with half baked justifications after the fact.

drallison• 15 years ago

The goal in language design is not "least astonishment" but inferability. And, while Python may not meet the least astonishment goal for new users coming from another language, Python is remarkably inferable and internally consistent.

njharman2• 15 years ago

Written by someone who actually knows Python (which is rare for articles like this).

The actual surprises section is great.

Python and the Principle of Least Astonishment

Related Stories