crdavidson

I wrote Spall, one of the lightweight profilers mentioned in the post. I loved the author's blogpost on implicit in-order forests, it was neat to see someone else's take on trees for big traces, pushed me to go way bigger than I was originally planning!

Thankfully, eytzinger-ordered 4-ary trees work totally fine at 165+ fps, even at 3+ billion functions, but I like to read back through that post once in a while just in case I hit that perf wall someday.

Working on timestamp delta-compression at the moment to pack events into much smaller spaces, and hopefully get to 10 billion in 128 GB RAM sometime soon (at least for native builds of Spall).

Thanks for the kick to keep on pushing!

Location: Seattle, WA

Remote: yes

Willing to relocate: no

Technologies: C, Go, Python, Rust, x86_64 + MIPS Assembly, SQL, DWARF + PDB, WASM/JS

Résumé/CV: https://colrdavidson.github.io/resume.pdf

Email: colrdavidson@gmail.com

I'm a senior systems-focused engineer with 6 years of experience, working on a range of problems, from linux kernel network-stack debugging, to raw/embedded flash drivers, to web backend infrastructure, to debuggers and profiling tools.

Since my last startup, I've designed and shipped a small product of my own, a high-performance profiler compatible with chrome://tracing but with dramatically better load times, frame rate, and deeper stats.

You can see a conference talk about it here: https://vimeo.com/776796857

Try the free web version here: https://gravitymoth.com/spall/spall-web.html

Check out the native desktop version here: https://gravitymoth.itch.io/spall

Ooh, hadn't seen @disable before, super useful to know! That'll be handy for doing toggleable build options for tracing / debugging.

I've been using Odin for about a year now, many of the pain-points I've had have just been knowledge gaps. Odin's docs and debug info have slowly gotten better over time, and little discord-community tips here and there have made a huge difference for my quality of life.

Ooh, I've seen this before on the Odin newsletter! https://odin-lang.org/news/newsletter-2022-11/

Exciting to see this get some spotlight. Was it entirely from scratch? Did you build it on SDL or Sokol or something?

So, spall has a JSON parser to help all the poor folks stuck using JSON or the chrome devtools profiler, who still need to keep using it.

For everyone doing new things, there's a binary format which is pretty straightforward.

The binary side supports begin and end events, which are defined here: https://github.com/colrdavidson/spall/blob/master/formats/sp...

and the header library mirrors those structs, and chalks out the interfaces around here: https://github.com/colrdavidson/spall/blob/master/spall.h#L2...

Basically, it's a small header, and then events, plopped one after another into the file. Each thread gets a buffer to store events as they come in, and they get written as atomic chunks with fwrite/append, so threads never have to interact directly.

In practice, the overhead for tracepoints with the binary format is super low, depending on the machine on x86, we've seen somewhere between 12-20ns for begin+end trace overhead when compiled with clang (+ occasional disk flush overhead, if you're generating a ton of data). I believe I can get that overhead even smaller to get things nice and lightweight, it's a major goal of mine to be able to trace 10+ million events per second without impacting runtimes significantly, so that bigger projects can be comfortably fully auto-traced.

Yep, core_net is the bulk of it, and I have a fork with changes to bring it up to date with master, we're just hashing out final details to get it ready to wrap and ship at this point.

Sure! So far, Odin has been awesome for doing custom allocation. Everything in the core library that allocates takes an optional allocator parameter, which means that you can use quite a few of the core libraries comfortably even in embedded contexts, and make it all sit on top of an arena in WASM's weird memoryland if you need to without an issue.

Outside of the WASM context where Odin runs into some difficult friction, systems-level work has been reasonable. I wrote a good chunk of a C/DWARF debugger in both C and Odin to compare, and Odin was by far the more pleasant language to write it in, my Odin code was much smaller because I didn't need xmacros, and I had proper namespaced enums, tagged unions, and slices, but still looked/read pretty similarly to well-written C or Go. I did have to wrap a few syscalls myself though, because the core library doesn't quite cover things like ptrace or networking yet.

I've put a fair bit of work into getting cross-platform networking into Odin's core library, hopefully we'll see it show up soon enough, and get some of those common nice-to-haves crossed off.

So, dtrace2spall is going to generate much smaller files and impact runtime less, because it samples with dtrace, and then converts the data to spall's format as a post-pass.

Auto-tracing is far more comprehensive, but it's a big datadump, and it does impact runtimes a bit more.

So far I've mostly used auto-tracing to figure out how bigger pieces of code work, rather than strictly for profiling. It was super helpful trying to walk through the more complicated parts of the Odin compiler to fix things, being able to see the order of functions called on the timeline.