Tracing HTTP Requests with tcpflow

https://ananthakumaran.in/2022/11/12/trace-http-requests.html

by ananthakumaran • 4 years ago

109 16 4 years ago

ananthakumaran.in

nhoughto• 4 years ago

This looks handy, better ergonomics than strace to just hook into http requests of a running process.

I wonder how it works with TLS if it’s working at the socket level..

youngtaff• 4 years ago

I’m guessing it doesn’t work with TLS as it’d need the keys to decrypt the traffic

LinuxBender• 4 years ago

I recall seeing a thread somewhere saying tcpflow would not add this capability and they point people to ssldump [1][2] and even that has some limitations.

[1] https://github.com/adulau/ssldump

[2] https://linux.die.net/man/1/ssldump

fmajid• 4 years ago

With TLS 1.3 and Perfect Forward Safety, even knowing the secret key is no longer enough. Otherwise, ssldump is a tool that would dump sessions if you knew the server private key and were somehow able to force negotiating a non-PFS TLS 1.2 cipher suite.

xtreak29• 4 years ago

There seems to be an issue open for this https://github.com/simsong/tcpflow/issues/58

imiric• 4 years ago

For a tool that understands TCP and HTTP conversations, the filter syntax is awfully similar to tcpdump's, and having to use `grep -A 15` to filter specific requests seems clunky.

It's good knowing this tool exists, but I think I'll stick to tcpdump and Wireshark. In Wireshark it's trivial to use the `http.request` filter to do this, and following the TCP conversation with decoded bodies, or specifying a TLS cert, is equally simple.

denysvitali• 4 years ago

For a CLI Wireshark, use tshark

ystad• 4 years ago

A thought. I would usually start with my application logs. Reverting to a tcp tool for a http app problem seems cobtrived

0x445442• 4 years ago

Yeah the app looked like it was delegating to another service and needed a bearer token for that. The app logs should have showed this straight away. Furthermore, I’ve been burned enough to know that if I have a service that’s dependent on other services then I setup tasks in my service to ping the health of those services periodically and log warnings/errors if they’re unreachable. This allows me to divert blame as quickly as possible if alerted in the middle of the night.

jimmyed• 4 years ago

There is also netpeek[1] which has better filtering capabilities and UX overall. It supports ngrep like bpf filters which we found useful.

1: https://github.com/darshanime/netpeek

_ZeD_• 4 years ago

in this space I have successfully used mitmproxy, charles and, with the best experience, fiddler

mitmproxy.org www.charlesproxy.com https://www.telerik.com/fiddler/fiddler-classic

rwestergren• 4 years ago

+1 for Fiddler, though it doesn't sound like the author was in a position to forward traffic to an HTTP proxy at the time in prod

thegabez• 4 years ago

This assumes you have shell and those tools installed via your Dockerfile. Best practice is to have a multi stage build and just have the working binary, keeping the image as light as possible.

So when do you make a fat image with debug tools and when do you keep it skinny?

mozman• 4 years ago

Or you’re running on a VM and can install these tools for debugging and then blow it away when you’re done

29athrowaway• 4 years ago

There's also termshark, a TUI for tshark inspired by Wireshark.

You also have mitmproxy which can be useful if TLS is involved.

aftbit• 4 years ago

Just because I know them better, I would have used `netstat -putln` and `tcpdump -i lo -n -A port 8000` to do the same thing. I'll take a look at tcpflow and ss though - it's always nice to know more tools.

Tracing HTTP Requests with tcpflow

Related Stories