'with' gets better with python 3.1

http://h3manth.com/content/statements-get-better-python-31

by init0 • 14 years ago

22 29 14 years ago

This is cool.. But of the things that 3.1 has got me excited about, this doesn't really move the needle. It's pretty straight forward to throw together a context manager that deals with multiple files and access using `files.in` and `files.out` if you want to.

masklinn6• 14 years ago

It's not that straightforward to create your own context manager, and creating a correct generic wrapper for the use case of composing multiple context managers is even harder (as demonstrated by the woes of contextlib.nested).

And if you're going to create a custom "composed" context manager every single time, you're probably better off just nesting `with` statements.

pejoculant• 14 years ago

Compound with statements are nice, put this is not a 3.1 feature. This was added in 2.7: http://docs.python.org/reference/compound_stmts.html#the-wit...

koenigdavidmj• 14 years ago

And you could hack this with contextlib.nested before.

archangel_one2• 14 years ago

Despite the numbering, 2.7 was released a year after 3.1. It added a bunch of features that helped compatibility with 3.x.

throwaway54-7622• 14 years ago

Thanks, I forgot about that.

briancurtin• 14 years ago

It was a feature to both. Check the same document for the 3.1 release (and the What's New).

adharmad• 14 years ago

The syntax seems very similar to Lisp's with-open-file macro.

adaszko• 14 years ago

Since you can write your own "context managers," the with statement analogue from Lisp world would rather be unwind-protect.

ufo• 14 years ago

Yes, basically the only difference is that Python's with operates over a context management protocol involving some `__magic__` methods so it can work with other things in addition to files.

masklinn6• 14 years ago

You can also do that in common-lisp by using the special form `with-open-file` is built upon: unwind-protect[0]. `with-open-file` is just a macro over it, specialized for files handling.

[0] http://www.ai.mit.edu/projects/iiip/doc/CommonLISP/HyperSpec...

sprash• 14 years ago

Since nothing got better with python 3.0 I'll stay with 2.x as long as I can.

rflrob• 14 years ago

Nothing got better with Python 3? What about the print function and true division? Sure, those are both there as from __future__ import's, but I'd definitely call those improvements.

I haven't switched to Py3k yet either, but I'm just waiting for a few more packages I use heavily to get the bugs worked out.

archangel_one2• 14 years ago

Also, dict comprehensions. You can fake them in 2.x, but the syntax is ugly and cumbersome.

masklinn6• 14 years ago

> What about the print function and true division?

Those are actually fairly minor in the grand scheme of things (and compound `with` statements have been backported to 2.7).

More interesting are:

3.0

* unicode strings by default and much better overall support of unicode

* chained exceptions (I don't believe they've been backported)

3.2

* argparse merged to the stdlib

* ElementTree 1.3 finalized in stdlib (3.2)

* functools.lru_cache, itertools.accumulate (nothing game-changing, but nice to have in the stdlib)

* threading.Barrier

* concrete tzinfo implementation (for fixed UTC offsets)

* contextlib.ContextDecorator (creates a type which works both as a contextmanager and a function decorator), and improved contextmanager support across the board (ftplib, popen, TarFile, ...)

3.3 (soon to be released)

* Virtual environments shipped by default

* Namespace packages

* Kind-of obscure, but flexible string representations (strings can dynamically and transparently switch their implementation between latin-1, ucs-2 and ucs-4, this kills the narrow/wide compile-flag shit and generally makes strings more space-efficient)

* sub-generator delegation, instead of:

    for item in sub_generator_function():
        yield item

  you can just write:

    yield from sub_generator_function().

  it also correctly forwards sent and thrown values (generator.send() and generator.throw()) to the sub-generator, so it actually translates into an unholy mess which now just works and allows more easily splitting a large function into sub-iterators.

* lzma/xz support (7z)

I'm actually excited for 3.3, there's plenty of cool stuff in there.

throwaway54-7622• 14 years ago

Nevermind that this feature isn't particularly exciting... Python 3.1 is "old." Python 3.2 has been out for a while, 3.3 will be coming out soon... there are plenty of interesting features in 3.x. This is a small part of the overall picture. Rock on, Py3k devs.

AceJohnny22• 14 years ago

I don't get it. Can someone explain to me how this is different or better than:

  infile = open('site.log'); outfile = open('err.log', 'w')
  for line in infile:
       if 'ERROR' in line:
           outfile.write(line)
  infile.close(); outfile.close()

Sure, you're saving a line with the files being automatically closed at the end of your scope, but hardly "MIND == BLOWN".

What am I missing?

endlessvoid94• 14 years ago

elegance?

baddox• 14 years ago

"MIND == BLOWN" is a hyperbole in almost every context, but the `with` statement is just cleaner code with fewer chances to screw up (e.g. by forgetting to close).

Maascamp• 14 years ago

Error handling.

graue• 14 years ago

You're missing that if an exception gets thrown, the files are still guaranteed to be closed as soon as you exit that 'with' block.

Imagine you open 'infile', but then opening 'outfile' fails. To prevent a resource leak, now you need to close 'infile'. The linked code guarantees this. Likewise if there is any error which aborts the copying, both files will get closed and you don't have to worry so much about corner cases.

See this for more:

http://effbot.org/zone/python-with-statement.htm

nnq2• 14 years ago

In a more complex application, an exception might be thrown before the last line that closes the file (and caught god knows where or not at all), so the files may not be closed... then again, if you actually try to catch the exceptions you get an extra level of indentation by using `with`.

More... "philosophy" here: http://stackoverflow.com/questions/11373502/zen-of-python-vs...

AceJohnny22• 14 years ago

Ah, so 'with' is a clearer syntax for exception handling, that's what I was missing. Thanks to you, masklinn and graue for clearing that up.

masklinn6• 14 years ago

> Sure, you're saving a line with the files being automatically closed at the end of your scope

That's actually pretty significant if you're, say, using Pypy or IronPython, otherwise you're leaking handlers if you've got an error raised.

But he was actually comparing the code to:

    with open('site.log') as infile:
        with open('err.log', 'w') as outfile:
            # code

. MIND=BLOWN is tongue-in-cheek. Hopefully. Then again, he's commenting on a 2 or 3 years old version of python so...

ef4• 14 years ago

I think "with" is a great example of where Python and Ruby differ philosophically. Python uses "with" to very explicitly implement their state manager protocol.

Ruby doesn't need "with" because "with" is really just a limited special case of what you can do with blocks.

The Python fans will counter that their less general solution is harder to abuse.

masklinn6• 14 years ago

> The Python fans will counter that their less general solution is harder to abuse.

Not sure why anybody would say that, you can plenty abuse `with` if you want to have fun, the only thing you can't do is "dual-use" the block really (grab a context and invoke the block multiple times). And that you could always do with a decorator, if you're going there. The only thing `with` is is less flexible/general.

`unwind-protect` or `BlockClosure#ensure:` are probably much better examples of the philosophy opposite python though, I've always found ruby rather half-assed.

jlarocco• 14 years ago

Not being mean, but this article kinda made me laugh. Python 3.1 has been out over 3 years[1]. And with the over the top "Mind == BLOWN"... Somebody should have told him about that release 3 years ago :-)

http://www.python.org/doc/versions/

'with' gets better with python 3.1

Related Stories