Ask HN: Easiest graphing / data viz solutions for graphing logs?

https://news.ycombinator.com/item?id=34006223
by vsroy • 4 years ago
1 2 4 years ago

I have a bunch of logs in structlog form (can easily be converted to JSON). Think:

``` timestamp=1 idx=2 key=foo key2=bar key3=4 rank=2 timestamp=2 idx=3 key=baz key2=box key3=4 rank=3 timestamp=3 idx=3 key=baz key2=box key3=4 rank=3 mode=start timestamp=4 idx=3 key=baz key2=box key3=4 rank=3 mode=end ```

and I'd like to do a bunch of adhoc / custom graphing. For example: there's the obvious stuff like graphing how many times I had the value "baz" for "key2" in "rank=3" per minute.

But there's also less obvious stuff like "graph the average time between mode=start/mode=end" and calculate the average delay between an "mode=end" and the next "mode=start" per rank.

So far I've been doing pretty well with a jupyter notebook + pandas, but it's pretty verbose, and I don't think matplotlib is the best graphing software. Also, pandas sucks.

Are there better solutions for graphing? I was looking into gnuplot, but its API seems unwieldy.

I'm looking for something lightweight (my logs are only on the order of ~ 50 - 500 MB). Not trying to setup Prometheus / Loki, or anything of similar complexity.

Related Stories

Loading related stories...

Source preview

news.ycombinator.com