Python wats

by luu • 11 years ago

123 122 11 years ago

ak217• 11 years ago

Of these, only += on tuple elements and indexing with floats strike me as WTFs, and many others are evidence of the author really going out of their way to look for behaviors to nitpick on. The absolute majority of these are things that Python programmers either will never encounter, or can easily explain from Python "first principles" (e.g. most of the nitpicks on string and sequence behaviors).

I can tell you what I consider WTF worthy - things that I and others have undoubtedly been bitten by because they are ubiquitous. Those are mutable defaults, lack of loop scoping, and "global"/"nonlocal" scoping. Oh, and add the crazy "ascii" I/O encoding default on 2.7 (and the inability to cleanly reset it) to that list.

cma2• 11 years ago

Bool acting as an integer instead of requiring a conversion (True//(True + True) == False*25).

ubernostrum3• 11 years ago

Python originally didn't have a boolean type, and people used integer 0 and 1 as stand-ins for boolean values. When the boolean type was added, first (before the type existed) "True" and "False" were added as built-in aliases for integer 1 and 0, and then bool was implemented as a subclass of int, locked down so only two instances (True and False, with integer values 1 and 0) could exist.

So while it looks weird there is a rational backwards-compatibility reason for it.

bpicolo4• 11 years ago

Yeah, things liked the nitpick on (1,2,3) == sorted((1,2,3)) seem not really `wats`. sorted returns a list, surprise surprise.

The only actual surprise on this list is, I agree, that += operator. Especially because tup[0] = tup[0] + [1] fails as expected.

aldanor3• 11 years ago

sorted is always a list, whereas reversed is an iterator (at least in Python 3)

tsujamin• 11 years ago

the fact that calling sorted(b) where b is a reversed object affects the following comparison "sorted(b) == sorted(b)" is wierding me out..

Ref: http://pastebin.com/VQs9GSzp

ngoldbaum3• 11 years ago

The += on tuple elements thing kind of looks like a bug in CPython.

bpicolo4• 11 years ago

Agreed. tup[0] = tup[0] + [1] fails as expected.

aldanor3• 11 years ago

It's quite obvious where it comes from though, it first calls __iadd__ on the list and that succeeds, and then it calls __setitem__ on the tuple which fails; however, the first operation already mutated the list.

This has nothing to do with tuples, I think, as one can constructs many other examples just like this one.

hueving2• 11 years ago

The indexing with a float was on a dict. If you can index with a string, why not a float? If it's hashable, it's good!

ubernostrum3• 11 years ago

Yup. If hash(x) == hash(y), then some_dict[x] and some_dict[y] will return the same value. The int/float is just a convenient red herring on that one, since it's an easy way to get two different-looking keys which hash the same, combined with the double red herring of the initial "can't index" example being a list (which does require integer values for indexing).

ketralnis5• 11 years ago

+= on lists is an annoying one:

    def myfunc(mylist):
        mylist += [1,2,3]

is NOT the same as:

    def myfunc(mylist):
        mylist = mylist + [1,2,3]

Try this on each of those definitions:

    a = [4,5,6]
    myfunc(a)
    print a

The first one mutates 'a', the second only reassigns (and discards) a temporary `mylist`.

Put another way, += is not just syntactic sugar. With lists, it actually mutates the original list.

pikzen• 11 years ago

Which is what every language with a C inheritance does. += has always mutated the original value, I don't know why you expected it to be different.

To me the real wtf is the second, where python creates a variable with the same name and discards it.

albertzeyer• 11 years ago

It is syntactic sugar to call mylist.__iadd__. The behavior to modify it inplace is common also for other non-pure data structures.

I'm not sure but I think that list.__iadd__ is even more tricky: When the new object still fits into the allocated array, it will modify it inplace, otherwise it returns a newly allocated list and doesn't modify it inplace. So you cannot rely on __iadd__ modifying a list (or anything else) inplace.

crdoconnor8• 11 years ago

Truthiness values are also WTF worthy (although not mentioned above).

    if not x:
        do_something()

Where do_something will occur if x is None, zero, 0.0, a blank string or even midnight in some versions of python.

This has bitten me multiple times. It's a violation of python's explicit over implicit. It's an example of weak typing in an otherwise strongly typed language. There's really no good reason for it.

IMO, if x isn't a boolean, that line should just throw an exception. If you want to check for x being 0.0 or "", you should do x == "" or x == 0.0.

josegonzalez• 11 years ago

Why not do `x == False` then? Arguing about a language not being as strongly typed when you're not being as explicit about your own types seems pretty roundabout.

StavrosK4• 11 years ago

I completely disagree. Your snippet above is exactly what I do whenever I want to check for falsiness. If I get a None, a 0, 0.0, "" or [], I check for it using "not x", sometimes using it for multiple return values (e.g. if I get a None or a 0 from mydict.get("param"), meaning that the value wasn't there or is 0).

To me, this is the absolutely most reasonable way to check for truthiness or falsiness. If you want to check for equality or identity, use == or is.

joshz• 11 years ago

Both behaviors are documented and "if not midnight" adds a fair bit of complexity, probably a good idea to know exactly what happens when you write that.

[1] https://docs.python.org/2/library/stdtypes.html#truth-value-...

[2] https://docs.python.org/2/library/datetime.html#time-objects

Veedrac12• 11 years ago

If Python's `if` is weakly typed, Rust's `for` is weakly typed. Few would argue for the later, so I disagree that this is weak typing.

aexaey• 11 years ago

3 different kinds of NaNs is pretty rad too. No idea what's happening here:

   >>> x = 0*1e400
   >>> set({x, x, float(x), float(x), 0*1e400, 0*1e400})
   set([nan, nan, nan])
   >>> set({x, float(x), 0*1e400, 0*1e400})
   set([nan, nan, nan])
   >>> set({x, float(x), 0*1e400})
   set([nan, nan])

EDIT: it gets worse:

   >>> x = 0*1e400
   >>> y = 0*1e400
   >>> z = 0*1e400
   >>> set({x, x, x})
   set([nan])
   >>> set({x, y, z})
   set([nan, nan, nan])

nv-vn• 11 years ago

It seems like these all are exclusive to dynamically/weakly typed languages. Anyone have examples of wats for languages with strong static typing?

Macha• 11 years ago

Java:

* Reassigning interned integers via reflection

* private fields work on a class level, not an instance level, i.e. Instances of a class can read private fields of other instances of the same class.

* Arguably package-private fields as the docs like to avoid mentioning they exist.

* ==, particularly in regards to boxed types.

* List<String> x = new ArrayList<String>() {{ add("hello"); add("world"); }};

* The behaviour of .equals with the object you create above

* Type Erasure

dlwh• 11 years ago

to add another:

    System.out.println(0.0 == -0.0); // true
    System.out.println(java.util.Arrays.equals(new double[] {0.0}, new double[] {-0.0})); // false

(It's documented in the contract for Arrays.equals, but still kind of ridiculous.)

tomsthumb• 11 years ago

Byte is signed. `Byte b = 127; b += 1; assert b != 128;`. That's not cool man.

isbadawi• 11 years ago

Java Puzzlers spring to mind. There's a book [0], and some fun talks available on youtube [1][2][3].

[0] http://www.javapuzzlers.com/

[1] https://www.youtube.com/watch?v=wbp-3BJWsU8

[2] https://www.youtube.com/watch?v=yGFok5AJAyc

[3] https://www.youtube.com/watch?v=Wnzyp1aitb8

Ao7bei3s• 11 years ago

Sure. Javas signed byte type comes to mind.

Oh, and `null`, in any language ;-)

Veedrac12• 11 years ago

Some guy made https://www.reddit.com/r/lolhaskell and put 3 posts there.

oct• 11 years ago

OCaml's parsing rules are pretty opaque sometimes. For example, if you have an "if" statement that's just performing effects (not returning a value), you may run into behavior like this:

    # let x = ref 0;;
    val x : int ref = {contents = 0}
    # let y = ref 0;;
    val y : int ref = {contents = 0}
    # if false then
        x := 1;
        y := 1;
      ;;
    - : unit = ()
    # (!x, !y);;
    - : int * int = (0, 1)
    # y := 0;;
    - : unit = ()
    # if false then
        let z = 1 in
        x := z;
        y := z;
      ;;
    - : unit = ()
    # (!x, !y);;
    - : int * int = (0, 0)

jaibot• 11 years ago

Okay, I don't get this one:

  >>> False == False in [False]
  True

julian37• 11 years ago

Both "in" and "==" are comparison operators that can be chained and so it's interpreted as

    False == False and False in [False]

https://twitter.com/marcusaureliusf/status/55794887300903731...

bpicolo4• 11 years ago

>> 1 in [1] in [[1]]

True

tcdent2• 11 years ago

I tend to write complex conditionals with parenthesis, sometimes excessively, but the clarity and guarantee of correct evaluation far outweighs any brevity.

    >>> False == (False in [False, ])
    False

ngoldbaum3• 11 years ago

I have a favorite NumPy wat:

   >>> import numpy as np
   >>> 'x'*np.float64(3.5)
   'xxx'

(on recent NumPy releases this raises a deprecation warning)

Clearly the correct answer is 'xxx>' ;)

aldanor3• 11 years ago

How about

>> type(np.uint64(1) + 1) numpy.float64

agf2• 11 years ago

I thought this was going to be about the "Investigating Python Wats" talk Amy Hanlon (my co-worker) gave at PyCon last year: https://www.youtube.com/watch?v=sH4XF6pKKmk

rhizome31• 11 years ago

Excellent talk. Her use of the dis module to explain wats by showing bytecode was particularly enlightening. It inspired me to give a similar talk next week end at PyConFr: https://github.com/makinacorpus/makina-slides/blob/master/py... (in French but the meat of it is Python code).

dalke3• 11 years ago

You might post this as an HN submission.

dllthomas6• 11 years ago

Working in Python, my biggest wat is that a test can cause a failed assertion in the code and still succeed (if AssertionError is caught somewhere upwards in the code, which will happen at any `except Exception`).

eru4• 11 years ago

That's why you should inject defects into your code, to see whether your tests catch them. There are tools to do that automatically.

dllthomas6• 11 years ago

I feel like I may be missing some of what you're trying to communicate. I agree that injecting defects can help test a test suite and that this can be a good practice. I don't quite see the relevance to the issue I described.

Edited to add - thanks for responding, by the way. I'm confused by all the down-votes.

ketralnis5• 11 years ago

I've hit this too, it would be really nice if AssertError didn't inherit from Exception. There's a whole Exception object hierarchy but it seems like none of the parents of Exception are actually useable for anything

dllthomas6• 11 years ago

That would partially solve this problem (it's far rarer to see bare excepts or `except BaseException`), but could cause others. The question there should be what is (or should be) intended when someone says `except Exception` - it's not clear to me whether that should or should not include AssertionError.

A more direct solution might be to have a means of storing off every assertion failure into a (clearable) list. The test harness could then check that list at the end of each test. If a particular test should succeed after triggering an assertion failure, it can check the list to make sure it triggered exactly the expected failures and then clear it.

jakeogh• 11 years ago

False as a valid fd: os.stat(False) (False == 0)

polemic• 11 years ago

I don't find many of these wat-worthy.

In python 2.x you could redefine True and False, and this is fun:

    >>> object() < object()
    True
    >>> object() < object()
    False
    >>> object() < object()
    True

But python3 dropped a lot of wat-ness.

This one is fun too: https://bugs.python.org/issue13936

eru4• 11 years ago

all and emptiness [...]

If all would work any other way, it would be seriously broken. (Given the rules for how to convert list to bool.)

hueving2• 11 years ago

The very first example is stupid. What sane language would assume a non-empty string should be converted to a False boolean?

msane• 11 years ago

isn't it doing the opposite? str(False) is the string "False", which bool() casts to True because it is a non-empty string; Non-empty strings are cast to True booleans.

This is much faster than checking if the string has some semantic meaning first, which would have to be localized.

ketralnis5• 11 years ago

That's exactly what the grandparent is saying

lqdc13• 11 years ago

actually almost none of these seem like wats to me.. Just what I would expect.

The one wat was adding a float to a huge integer. Granted, I assumed something fishy might happen with such large numbers, so I steer away from doing those operations.

khulat• 11 years ago

Well to be fair it is only a wat if you don't know about floating point imprecision. Because that is exactly what is causing this contrary to what is stated. Also python just gets this behavior from C, so the other claim that this is a special case of python is also not true.

Since the 1 is shifted 53 bits to the left the number is right at the start of the number range where only even numbers can be represented.

You get 900719925470993 as the decimal number which you cast to float : 900719925470992.0

Then you add 1.0 and get 900719925470992.0 because of floating point imprecision and rounding. The next floating point number would be 900719925470994.0.

92 is less than 93 and this gets you this seemingly weird x+1.0 < x = true.

Veedrac12• 11 years ago

It's still unfortunate that `x + 1.0` rounds incorrectly. Python is one of the few languages that has correct (infinite-precision) float-int comparisons, so it's reasonable to expect addition to work similarly.

That said, I'm not aware of a language that does better, and I'm aware of many that do much worse.

misiti3780• 11 years ago

I thought I knew python .... I guess I do not know python ...

awesome idea - thank you

jayflux• 11 years ago

Can someone explain why the first one is a wat? To me it seems perfectly logical

Cast false to a string, then check the Boolean value of that string. It's not an empty string so of course it would be true.

lostmsu• 11 years ago

Because people from normal langs or without cs background expect type cast roundtrip to return original value if temp type can represent any value of source type (e.g. string can represent any value of bool)

StavrosK4• 11 years ago

So someone would expect that an int cast of "2" would be 2? What would an int cast of "a" be?

yokohummer72• 11 years ago

I interpret "wat" in the original "wat" presentation as "certain behaviors that can't be predicted by reading the code". In this regard, I don't think many things pointed out in the article can be considered as "wat". I could predict the behavior of many items fairly easily. The same thing could not be applied when I watched the original "wat" presentation.

The only exception I think is the bool coercion. Even though I can predict the behavior, I find it fairly disturbing. I believe that should have been removed when transitioning to Python 3. It is very unfortunate, considering Python is generally reluctant to implicit type conversion.

746F7475• 11 years ago

To me the first two: "Converting to a string and back" and "Mixing integers with strings" aren't "wats" at all.

ksikka• 11 years ago

Anyone know the story behind the converse implication operator? What it's useful for, and why it's undocumented?

mxmul• 11 years ago

Remember that booleans are integers in python. This is just the regular exponential operator.

johnsoft• 11 years ago

∗∗ is the exponentiation operator. True is implicitly converted to 1, and False to 0. It's a coincidence that 0^0, 0^1, 1^0, and 1^1 line up with converse implication.

asgard1024• 11 years ago

Maybe someone can explain better, but I don't think it's a coincidence.. In category theory, there is a general notion of "exponential object". This works like normal exponentiation in various categories of numbers, and like modus ponens (and also reversed - given exponent and power, you can deduce the base) in various categories of logic.

Veedrac12• 11 years ago

There's no coercion - True is just an alias for 1, and False for 0. True <pow> False is just an ugly way of writing 1 <pow> 0.

chris_wot• 11 years ago

How are the converse implication operators worthy of a wat? That's completely normal...

pvdebbe2• 11 years ago

Besides the mentioned debunks of wat, the

   all([[[]]]) == True

makes perfect sense, because the all all does is check the truthiness of the argument's elements. An empty list is false, non-empty is true, no matter the contents.

agf2• 11 years ago

By itself, yes, but

  >>> all([[[]]]) == True
  True
  >>> all([[]]) == True
  False
  >>> all([]) == True
  True

is pretty counter-intuitive at first glance.

pvdebbe2• 11 years ago

At first glance, yes. Needs a moment to think this through. But not a wat because this works exactly as the logic says.

If we denote

  x = []

then

  bool(x) == False    # empty list is falsy
  bool([x]) == True   # nonempty list is truthy
  bool([[x]]) == True # nonempty again

and finally,

  [bool(elem) for elem in [[x]]] == [True]  # all True!

which is the thing `all' is interested in. It is more like a newbie mistake or careless document reading if the user thinks `all' runs through the lists and all of the nesting too.

fela2• 11 years ago

the behavior of is seems pretty hard to predict

  >>> 3.1 is 3.1
  True
  >>> a = 3.1
  >>> a is 3.1
  False

fela2• 11 years ago

It seems to be a case of leaky abstraction, where the way python caches small objects becomes visible, another example:

  >>> a = 'hn'
  >>> a is 'hn'
  True
  >>> a = ''.join(['h', 'n'])
  >>> a is 'hn'
  False
  >>> a
  'hn'
  >>> a = 'h' + 'n'
  >>> a is 'hn'
  True

Edit: found another interesting case

  >>> a = 1
  >>> b = 1
  >>> a is b
  True
  >>> a = 500
  >>> b = 500
  >>> a is b
  False

odonnellryan• 11 years ago

Second case is because of the range of integers that are cached, right?

Veedrac12• 11 years ago

`is` means "do these things reside in the same location in memory?", and the runtime is merely required to give a plausible answer.

For inequivalent values or `True`, `False` and `None`, the result is defined. Identity is also preserved across assignment. For everything else, the answer is "whatever the runtime feels like". PyPy, for instance, will always say True for integers and floats.

It's not just hard to predict - it's inherently unpredictable. The runtime gets free reign as long as you can't prove that it's lying.

avinassh• 11 years ago

Not sure if I can call this as `wat`, but this one puzzled me at first:

    foo = 1.0 + 2 # Foo is now 3.0
    foo = 1,0 + 2 # Foo is now a tuple: (1,2)
    foo = 3 # Foo is now 3
    foo = 3, # Foo is now a tuple: (3,)

source - https://wiki.theory.org/YourLanguageSucks#Python_sucks_becau...

Retra• 11 years ago

The ',' in always creates a tuple. Why is that puzzling?

amelius• 11 years ago

I always run into trouble in Python with closures and assignment. E.g.,

  def f():
      x = 0
      def g():
          x += 1
      g()
  f()

Veedrac12• 11 years ago

Here's a quick interpretation of Python's method:

Any assignment operator (compound or not) follows the same namespace rules. Scope lookups are consistent within a scope, regardless of order. Any "namespaced" lookup (`x[y]` and `foo.bar`) will act inside that namespace.

Non-namespaced identifiers will always assign into the nearest enclosing scope (a scope is always a named function) by default, since clobbering outer scopes - particularly globals - is dangerous. You can opt in to clobbering with `global` and `nonlocal`. If you don't ever assign to a name in the current scope, lookups will be lexical (since a local lookup cannot succeed).

---

Hopefully most of these value judgements should be relatively obvious.

ris• 11 years ago

I would argue that the vast majority of behaviours described here are precisely the desired behaviour.

In fact, I think the ones he picks reflect tremendously on his skill as a programmer.

Take for instance his apparent insistence that int(2 * '3') should produce 6. Cool. Magic "special" behaviour for the multiply operator if the string happens to contain a number.

The same goes for his desire for the string "False" to be falsey. Wonderful magic special behaviour dependent on the (possibly unknown) content of a string means I've got to go round checking my strings for magic values.

d23• 11 years ago

I agree, but there's enough weird shit going on here that it's still a worthwhile "article". For me, the `mixing numerical types`, `Operator precedence?`, `extend vs +=`, `Indexing with floats`, `all and emptiness`, and `Comparing NaNs` were all interesting.

Python wats

Related Stories