LTTng: An open-source tracing framework for Linux

by thezorg • 12 years ago

68 15 12 years ago

dmix2• 12 years ago

Wow, open source sites are getting nice designs these days. Is there a business behind this project or is it via donation?

The documentation is excellent.

cbab3• 12 years ago

Thanks, we worked quite hard on this new redesign!

edit: Yes, EfficiOS [1] is the open source business behind the LTTng project :).

[1] https://www.efficios.com/

dmix2• 12 years ago

Some UX feedback after attempting to install it on Fedora:

1) Clicked Fedora on install docs

2) Installed lttng-tools and lttng-ust RPMs

3) Clicked Getting Started in docs. Ran "lttng list --kernel".

3a) Got an error saying I need to run the systemd service.

4) Started systemd service (didn't see this mentioned in the docs?)

5) Reran "lttng list --kernel".... error saying I'm missing a module

5a) Reread docs, realized I need to build LTTng-modules. Which led me to find out I need to also build liburcu

6) Installed liburcu but got an error building lttng-modules https://gist.github.com/dmix/d2d4c3c1b823e82486bf (I could submit this as a bug report?)

So I gave up to attempt the install another time.

jordigh• 12 years ago

Here is a pretty cool presentation about it, in French:

https://www.youtube.com/watch?v=9XokHJadkq4

The slides are in English, if that helps:

http://lttng.org/files/papers/presentations/lttng20tracingfo...

cbab3• 12 years ago

You can also find all the presentations/publications that we did through the years here [1].

[1] http://lttng.org/publications/

hitlin372• 12 years ago

how different is this from oprofile?

abusque• 12 years ago

Well, I've never used OProfile myself, so please take this answer with a grain of salt, but from a quick perusal of their website, and as the project's name seems to suggest, OProfile is a sampling profiler, which usually means it will collect data from counters at given intervals of time, producing a number of statistics over time. This generally implies a somewhat significant overhead, and consequent performance drop.

LTTng, on the other hand, is a tracer, which means it collects events from the kernel (using the built-in tracepoint facilities) as they happen. It's also possible to trace userspace apps (with lttng-ust), or define your own tracepoints. This has the benefit of being much more detailed, and also has a much smaller overhead.

You might be interested in reading the "What is Tracing?" section of the lttng docs [0], which does a far better job at explaining this than I do.

[0] http://lttng.org/docs/#doc-nuts-and-bolts

hitlin372• 12 years ago

thanks for explaining. it sure looks interesting. i'm going to read now.

krakensden• 12 years ago

oprofile has been abandoned, and no longer works on recent distributions. 'perf', I think, is the usual replacement, though the tooling is not as nice.

rasur• 12 years ago

So how does this compare to somerthing like Sysdig, for example?

mvlad• 12 years ago

"Excluding proprietary solutions, a few competing software tracers exist for Linux. ftrace is the de facto function tracer of the Linux kernel. strace is able to record all system calls made by a user process. SystemTap is a Linux kernel and user space tracer which uses custom user scripts to produce plain text traces. sysdig also uses scripts, written in Lua, to trace and analyze the Linux kernel.

The main distinctive features of LTTng is that it produces correlated kernel and user space traces, as well as doing so with the lowest overhead amongst other solutions. It produces trace files in the CTF format, an optimized file format for production and analyses of multi-gigabyte data. LTTng is the result of close to 10 years of active development by a community of passionate developers. It is currently available on all major desktop and embedded Linux distributions."

Further down you'll discover that your application(s) are linked with a library provided by the project.

Most of them rely on kprobes [1], or like ftrace on [2]. But there is also oprofile [3], and almighty perf [4], which rely on CPU extensions to implement tracing, (also implemented and by the HW manufacturer as well -- but in a OS-agnostic way).

[1] http://lxr.free-electrons.com/source/Documentation/kprobes.t...

[2] http://lxr.freeelectrons.com/source/Documentation/filesystem...

[3] http://oprofile.sourceforge.net/doc/internals/index.html

[4] https://perf.wiki.kernel.org/index.php/Main_Page

jpfr• 12 years ago

LTTng is closely related to liburcu (now on http://urcu.so/).

It provides lockless datastructures for screaming fast concurrency on multicore machines. You will be pleasantly surprised by the concurrent hash-map implementation that comes with it [1].

[1] here's the paper for background: https://www.usenix.org/conference/usenixatc11/resizable-scal...

wrl• 12 years ago

In the event LGPL is too restrictive, there's also concurrencykit (http://concurrencykit.org/), which provides similar mechanisms (both for concurrent memory reclamation and various datastructures built on top).

LTTng: An open-source tracing framework for Linux

Related Stories