Tracing Python

https://blog.koehntopp.info/2023/03/14/tracing-python.html

by jimyl • 3 years ago

104 13 3 years ago

blog.koehntopp.info

sys_64738• 3 years ago

A more traditional debugger for Python.

https://github.com/inducer/pudb

Spivak• 3 years ago

Fantastic article and something I've been looking for better tools for. None of them seem to have my ideal workflow though -- providing an external manifest of what and how to trace that can be used against an existing program without changing the code. Something like Otel's auto instrumentation.

peterkelly• 3 years ago

Python has a built-in API for tracing: https://docs.python.org/3/library/sys.html#sys.settrace

Here's an example of how to use it (obviously a minimal case, you can customise your handler do do whatever you need):

    def mytrace(frame, event, arg):
        if event == "call":
            print("call", frame.f_code.co_name, frame.f_locals)
        elif event == "return":
            print("return", frame.f_code.co_name, arg)
        return mytrace

    import sys
    sys.settrace(mytrace)

For example, given the following code:

    def three(z):
        return 3

    def two(y):
        return three("BBB") - 1

    def one():
        return two("AAA") - 1

    one()

It will output:

    call one {}
    call two {'y': 'AAA'}
    call three {'z': 'BBB'}
    return three 3
    return two 2
    return one 1

taeric2• 3 years ago

I confess I'm surprised the builtin wasn't one of the demoed options. Curious what the reason is.

isotopp2• 3 years ago

I just pushed an addendum, and it is being rendered right now.

yi_xuan• 3 years ago

Pdb also employs this approach for debug. I had wondered about pdb's implementation, when I saw it, I felt a little surprised.

RJIb8RBYxzAMX9u• 3 years ago

Wow, I can't believe I didn't know about this until now! I'm the lucky 10k[0] today.

[0] https://xkcd.com/1053/

lathiat• 3 years ago

There are many different tools, but when I want to quickly understand what the code was doing I have been reaching for pysnooper: https://pypi.org/project/PySnooper/

Has a pretty nice and importantly colored output by default, which I find much easier to follow. In many cases snooping one specific function has done the job. It has an easy to use decorator to decorate any function without having to otherwise intercept the main python invocation or set something up at the start. Good for libraries etc.

pysnooper has mostly been enough for me but the "snoop" mentioned in the article seems to be a superset and more. Will have to give it a go next time.

Outside of this if I want to understand performance or more roughly what code paths are executing (and not a per-statement trace) then austin and py-spy are both amazing.. convert to flamescope and drop into flamescope.app.

nomel• 3 years ago

What's the modern practical use case for tracing in python? Environments without an IDE/remote debugger?

I can't recall the last time I needed to enable tracing, in the last couple of decades, besides bash scripts.

isotopp2• 3 years ago

Originally the question was "What is Honeycomb.io and Opentracing doing and how can I get this into my existing application?" on IRC. On explaining, the discussion went into various directions (Fred Fish C Debugging Macros and tracing Python).

knlb2022• 3 years ago

I'll throw in my own too: https://github.com/kunalb/panopticon

chrisshroba• 3 years ago

Fyi, the table of contents gets cut off (its box is smaller than its contents) at zoom levels of 90% or larger for me. Here's a screenshot at 100% zoom https://imgur.com/a/0t9cKs5

Great article!

Tracing Python

Related Stories