Python Language Summit: PEP 654 – Exception Groups and Except

https://pyfound.blogspot.com/2021/05/the-2021-python-language-summit-pep-654.html

by BerislavLopac • 5 years ago

91 33 5 years ago

pyfound.blogspot.com

Recently I came up with a related idea for repeated error handling code, in the usual Python vein of introducing an abstraction to save 2-3 lines:

    try:
        foobar
    delegate handle_stuff:
    delegate handler_factory(something):
    except Foo:
        ...
    else:
        ...
    finally
        ...

Where "delegate handle_stuff" means that for any exception handle_stuff is called with the exception tuple and the exception is considered handled if it returns truthy. "delegate" of course takes an expression so that you can have Proper Adult Fun Handling Exceptions.

mfgs• 5 years ago

How is that ultimately different from handling the exception under `except`?

coldtea• 5 years ago

"in the usual Python vein of introducing an abstraction to save 2-3 lines"

formerly_proven4• 5 years ago

The "delegate" statement encapsulates which exceptions are handled and how in the supplied callable expression. This is functionally different from repeating "except (FooExc, BarExc, BazExc) as exc: handle(exc)". It would also save two to three lines on average.

d0mine• 5 years ago

  async with asyncio.TaskGroup() as g:
    g.create_task(concurrent_task1())
    g.create_task(concurrent_task1())

reminds me of the structural concurrency:

  async with trio.open_nursery() as nursery:
    nursery.start_soon(myfunc)
    nursery.start_soon(anotherfunc)

> nurseries ... rather a new control flow primitive that's just as fundamental as for loops or function calls

https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

Implementing a better task spawning API in asyncio, inspired by Trio nurseries, was the main motivation for this PEP https://www.python.org/dev/peps/pep-0654/#motivation

BiteCode_dev3• 5 years ago

It is explicitly inspired by it.

meowface• 5 years ago

Indeed, the screenshot at the bottom with that syntax says "inspired by Trio".

agumonkey2• 5 years ago

IMO this is a worthy language agnostic topic, modeling and managing execution tree error cases has ever been an issue for my brain.

_ix• 5 years ago

I’m a bit of a pythonista, but I haven’t worked on anything very complex for a couple of years. Maybe my experience is still a little shallow in SWE topics, but can you reify this a bit? What kind of situations have you run into where handling multiple exceptions would have been handy? Maybe it’s telling that I’d have to admit that I’m not sure how big a difference there is between error code returns and exception handling.

agumonkey2• 5 years ago

It's just that the diagram shown in the article/blog were more inspiring than 99% what I heard about error abstractions on any language (and yeah it's not about python only). I just wished to read more about very general solutions on nested/stacked calls with exceptions and how to reduce the perimeter and make things more readable (honestly most of the time exception handling for me was logger.error or wrap/rethrow.. nothing exciting)

question000• 5 years ago

This is exactly the type of thing that should be in PEPs right now.

It's practical, its forward thinking addressing an existing problem that's really prevalent in the py3 native async features and it's not incremental it addresses the problem in its totality.

Mathnerd314• 5 years ago

So this is replacing one cumbersome and unintuitive syntax (MultiError.catch) with a slightly shorter syntax (try except*). Given the amount of verbiage in the PEP and this blog post I'm guessing it will still be unintuitive though.

luhn• 5 years ago

The main advantage of this PEP over MultiError (IMO) is that standard except blocks will Just Work. For most use cases that will be sufficient, and those people will never have to think about this PEP.

BiteCode_dev3• 5 years ago

Not just slightly shorter, because you need a handler for with MultiError.catch, which will be a function with and if and an instance check.

Also, it doesn't give you easy "finally" and "else" clauses.

And "except *" calls the same handler for single and multi-errors, so for most cases, you avoid code duplication.

Plus, having a natural way to deal with this make the code easier to scan, not to forget IDE already have good support for a try/except, and wrapping things in it.

It's 100% a win.

carapace• 5 years ago

This PEP is a footgun. Remember that exceptions are GOTO.

If your exception-handling code is complex enough to be at all tricky you have a state machine and it behooves you to make that ad hoc machine explicit.

Do not implement a multiplexing GOTO.

kdmccormick2• 5 years ago

I see this sentiment a lot in language discussions; I really think the exceptions==goto parallel is overstated.

Exceptions are a recognition of and abstraction over a common pattern from exception-less languages: "if <error state>, return early with a special value and some info on the error". Client functions follow a pattern of "if <error returned>, then handle the error or propogate it upwards".

If you factor out that chain of early returns, you get "throw"/"raise". If you factor out the error handling, you get "catch"/"except". And if you turn that error info into an object/ADT, then you get the "Exception" class.

Exceptions are not adding anything new; it's syntactic sugar over a more verbose pattern you'd probably be using anyway. You could totally implement a clunky but logically equivalent version of exceptions yourself in any language.

Now, that being said, there's definitely a case to be made for avoiding over-abstracting and preferring explicitness to implicit magic. But that's much different than saying that exceptions are a under-abstraction like GOTO.

R0b0t13• 5 years ago

I routinely encounter places where I am forced to use try..except where it would be better and more easily understood if I did not have to work around the fact I am forced to goto out of the current scope.

wvenable• 5 years ago

Remember that exceptions are GOTO.

This is not true and I wish people would stop repeating this falsehood. If you had multi-valued returns and propagated every single error at every call site you will have implemented exceptions exactly.

jcelerier• 5 years ago

If your exception-handling code is complex enough to be at all tricky you have a state machine and it behooves you to make that ad hoc machine explicit.

1980: "that error handling code is a tedious and repetitive state machine, how could we abstract it ? ... let's create exceptions !"

2020: "exceptions implement a state machine, that state machine should be visible error-handling code ! let's remove exceptions and make error handling explicit"

2060: "that error handling code is a tedious and repetitive state machine, how could we abstract it ? "

BiteCode_dev3• 5 years ago

Exceptions are not a GOTO:

- they cannot make you end up any place in the code, only up the stack.

- going up in the stack, it leads to the destruction of any frame in the way.

- the exception object carries metadata about the origin and propagation.

- calling code can subscribe to the event and act on it. Handling is decoupled from raising.

- calling code can interrupt (in multiple spots) or stop the propagation.

aranchelk• 5 years ago

Many excellent points. I had never encountered the idea of exception=goto. My initial thoughts:

1) In the case of goto, behavior is determined in the invocation. Conversely, when throwing an exception, the behavior is determined by the surrounding try block, which very is often well outside the scope of that throw. In this sense they seem like polar opposites.

2) You can model exceptions as monads: create a wrapper type that can either hold a regular value or a failure. Then create a helper function that processes these wrapped values. When encountering a wrapped regular value, the helper unwraps it and runs the next computation; on encountering a failure the helper skips the next computation and passes on the failure. The computations take unwrapped regular values as input, but they return the wrapped type. Not only is this how exceptions are done in pure functional languages, to my mind it’s the most implementation-independent way to think about what they actually are. This description bears no resemblance to goto.

tsimionescu• 5 years ago

Exceptions are just as much goto as return is. They are just a jump up the stack.

ab111111111• 5 years ago

Fantastic. An N dimensional error-handling space. That's exactly what Python is lacking.

orf• 5 years ago

It's definitely lacking it, but I guess this comment is sarcastic. So what do you think Python is lacking?

Recursing• 5 years ago

As mitsuhiko noted[0], I fear this might break a lot of existing generic exception handling code :(

Am I wrong? Please someone tell me I'm wrong

https://twitter.com/mitsuhiko/status/1364257824759504896

BerislavLopac• 5 years ago

I'm not sure what he meant by that comment. ExceptionGroup is an Exception subclass, so `except Exception` should work perfectly fine.

Python Language Summit: PEP 654 – Exception Groups and Except

Related Stories