C Runtime Overhead

by lunixbochs • 11 years ago

167 31 11 years ago

falcolas• 11 years ago

Now imagine the speed if you could get rid of the syscall overhead. You would have to write a lot more code (and it probably wouldn't run on Linux due to trying to access protected memory locations or IO ports), but it would be significantly faster.

Aah, the promise of the Exokernel[1].

[1] http://u.cs.biu.ac.il/~wiseman/2os/microkernels/exokernel.pd...

pjmlp• 11 years ago

Well, when I started programming, C and Pascal execution speed were looked down like managed runtimes are still nowadays.

Serious application development for consumer computers was done in straight Assembly. Anything higher than that was "scripting". :)

scott_s• 11 years ago

Back when digital, electronic computers were new, assembly was called "automatic programming".

agumonkey• 11 years ago

Funny that I felt the scripting side of while reading some projects like wmii, or some plan9 utils. I'm under the impression that in UNIX early days, C was almost a rapid prototyping / scripting language.

blt3• 11 years ago

The promise of TempleOS!

malkia• 11 years ago

And The Interrupt List!!! (Ralph Brown's)

SixSigma• 11 years ago

The HPC guys use OS-Bypass to avoid system calls. [0]

Ron Minnich presented a paper at the Plan9 conference

Using Currying and process-private system calls to break the one-microsecond system call barrier [1]

Which is a technique he's been using on Blue-Gene. There are some more papers linked from the IBM Research website. [2]

[0] http://www.networkworld.com/article/2317965/tech-primers/os-...

Sorry the conference repository server is awol, all I could find is this :

[1] https://www.yumpu.com/en/document/view/7777923/using-curryin...

[2] http://researcher.watson.ibm.com/researcher/view_group_pubs....

lunixbochs5• 11 years ago

I had to implement buffered IO due to syscall overhead (the CTF solution called fgetc() and relied on libc's buffering for speed). My first working non-buffered test took on the order of seconds and made several thousand syscalls. When I mentioned optimistically buffered IO, that was "just flush on EOF, don't even look for newlines". Reducing the syscall count definitely helped.

1019142• 11 years ago

"...huck the whole thing and rewrite it in assembly."

What are the functions in libc that you truly need and use?

How many of those routines are you incapable or unwilling to rewrite yourself?

I have asked myself these questions on many occasions.

The cruft laden operating system I use (which I imagine by many people's standards is "bare bones") is hopelessly dependent on "libc". But I find that the userland programs I truly need can be written using only a minimal subset of the many functions that are compiled into it.

Could I write any of these in assembly and create my own "libasm" to do the syscalls? This is a project I have contemplated many times.

To motivate myself to keep writing more assembly in the age of scripting language du jour madness, I write a small program in assembly language and then write the same program in C and then step through each compiled program (e.g., with ald). Seeing some of the overhead is an effective motivator.

tptacek• 11 years ago

You for the most part don't need to rewrite anything. You can take the source code for most libc functions and compile them independently.

justincormack3• 11 years ago

Musl libc has every function in a separate file, so if you statically link you just get what you use, modulo a number of functions that use each other. So building it yourself is not going to be much better.

blt3• 11 years ago

It's sometimes startling how major foundational tools and systems ignore bloat.

For example, I wanted to use C++11's std::array to return fixed-lengths arrays by value on the stack. This was in a minimal API that originally included NO other headers. In MSVC, std::array included a tree of 100 other headers including gems such as istream, exception, new, malloc, and float.

Yes, I'm aware that none of this stuff gets compiled into the binary, but it shows a disregard for lean dependencies that is emblematic of a bigger problem.

StephanTLavavej2• 11 years ago

From our (STL maintainer) perspective, <array> includes <algorithm> because it needs equal(), <iterator> because it needs reverse_iterator, and <tuple> because it needs to provide get(). Minimal!

But yeah, it drags in a whole tree of headers. Minimizing this is possible, and we've taken steps to do so in the past, but it's a lot of work for possibly minimal benefit - user translation units tend to drag in many STL headers, especially if they're using precompiled headers like they should. We think our time is better spent fixing correctness bugs and implementing features.

eloff• 11 years ago

It's interesting to note that Go doesn't depend on libc. However, I rather doubt the Go runtime bootstraps any faster than libc, so it probably wouldn't help here. One consequence is that Linux and libc are tightly coupled and some of the "syscall" behavior you depend on is actually implemented as a wrapper in libc. This has caused some behavioral differences between Go and C that have surprised the language developers.

ufo• 11 years ago

can you give an example?

justincormack3• 11 years ago

Linux has very non Posixy ideas about threads, which are basically processes with shared memory. libc is supposed to present a Posix compatible interface. See [1] for some examples. Go simply exposes raw syscalls.

[1] http://ewontfix.com/17/

zokier• 11 years ago

It also interesting to note that in contrast Rust stdlib does depend on glibc. It does not even work on musl, that is on the wishlist: https://github.com/rust-lang/rfcs/issues/625

Of course you can use Rust without stdlib, but that leaves you with fairly barebones language.

lunixbochs5• 11 years ago

My pre-C solution was Go. It bootstrapped pretty fast, but I actually ran into a problem where my large embedded data structures (hash table, bloom filter, etc) took a very long time (hundreds of milliseconds?) to initialize. I guess it has a startup cost for embedded data. Go also has a slightly larger binary size, which may or may not have affected startup time on the CTF worker boxes.

justincormack3• 11 years ago

Static linking to Musl libc is pretty minimal, not sure you really need assembly

strace -tt /tmp/test

15:38:33.245549 execve("/tmp/test", ["/tmp/test"], [/* 16 vars */]) = 0

15:38:33.246436 arch_prctl(ARCH_SET_FS, 0x601070) = 0

15:38:33.246617 set_tid_address(0x6010a0) = 17227

15:38:33.246875 exit_group(0) = ?

15:38:33.247060 +++ exited with 0 +++

lunixbochs5• 11 years ago

My solution in the final trace was a minimal (300 sloc) libc with a total of two lines of assembly (to bootstrap syscalls) and didn't do any syscalls beside read, write, and exit. I've also toyed with the idea of testing exit via error (SIGSEGV, SIGILL, etc) because cleanup might be faster on a forced exit.

tedunangst• 11 years ago

Something funny about the numbers here. The assembly stub took 8ms, but the whole program only takes 1ms?

darshan• 11 years ago

That was a typo -- the assembly stub took about 0.8 ms as shown in the trace.

lunixbochs5• 11 years ago

Thanks, fixed.

neonscribe• 11 years ago

Micro-benchmarks lead to optimizing at the wrong level. In the real world, if process startup time is a bottleneck in your application then you are creating too many processes too quickly.

lunixbochs5• 11 years ago

You're absolutely right about the majority of the time in the real world. A ctf level is a contrived case, but that doesn't make this any less interesting for me. This was mostly a challenge of "how fast can I run my existing program?" and I think I nailed it pretty well with lib43 [0] (which is a working extremely minimal libc and crt0 replacement in about 300 lines of code).

My favorite part is my x86_64 linux syscall interface, in which I define a short assembly stub (syscall.c) and simply list syscall names (syscall.list) to wire them up. The calling convention between user-space functions and the kernel was close enough I was able to reduce the inline asm to two instructions (`mov rax, syscall num; syscall;`).

[0] https://github.com/lunixbochs/lib43

oso2k• 11 years ago

This is cool. I've been working on something similar[0]. It only takes a few more lines to add some of the other useful libc/posix/sysv things. My eventual goal is to rewrite asmutils'[1] httpd [2] in C and get a binary about 2-3x in size (2-3K).

[0] https://github.com/lpsantil/rt0

[1] http://asm.sourceforge.net/asmutils.html

[2] https://github.com/leto/asmutils/blob/master/src/httpd.asm

C Runtime Overhead

Related Stories