Dyad: Minimal, portable async networking library for C

by api • 12 years ago

97 46 12 years ago

curiousDog• 12 years ago

Very neat! Although after using C#'s TAP, callbacks sound like hell :)

regi3• 12 years ago

Callbacks can be avoided using co-routines. I started a project that uses asynchronous sockets but they appear synchronous. With a bit of abstraction, it's really easy to develop network applications even in C:

https://github.com/reginaldl/librinoo

brown-dragon3• 12 years ago

Your approach looks very interesting - the examples are definitely much easier to read and understand over most async C code I have seen. How does it work internally?

baruch7• 12 years ago

Didn't find this one before.

I wrote my own user-space-threading library called libwire. Mostly for not liking the malloc-everywhere approach so common everywhere. The tradeoffs are different and the code is more verbose at times but I like the fact that there are no mallocs in code, at least not very explicit ones. I do provided a memory pool so it allocates memory but it is bounded in size.

libwire: https://github.com/baruch/libwire

list of coroutine/user-space-threading libraries: https://github.com/baruch/libwire/wiki/Other-coroutine-libra...

NkVczPkybiXICG• 12 years ago

It uses select, which is notorious for poor performance (although is relatively portable). Try not to use this project for anything high-load.

regi3• 12 years ago

And anything that requires more than 1024 fds...

baruch7• 12 years ago

You can increase the define if you need more than 1024 fds but then the performance is getting very poor.

Chromozon• 12 years ago

How are you supposed to do socket programming without select?

BruceM• 12 years ago

poll, epoll, kqueue, sigio and other options depending on OS, etc.

deathanatos• 12 years ago

poll is fairly portable, but still has scaling problems. To get beyond scaling problems, you usually need something OS-specific, such as epoll (Linux), kqueue (BSDs of some kind), etc. One of the things I'd expect of a cross-platform networking library would be to use the best available abstraction, especially on major platforms such as linux, and perhaps fall back to select on odd platforms. Of course, your docs need to point out what the behavior is.

The problem with poll/select is that you need to pass (transfer) to the kernel the entire list of events you're interested in, just to wait for a single event. Each time you re-enter your event loop you need to do this, and if you have thousands of connections, it will break down, because poll is O(number of connections).

epoll, for example, works around this by having an "epoll FD", which on the kernel side contains all the events you're interested in. You wait by just waiting on that epoll fd with epoll_wait, but you don't specify the events you're interested in when you wait: you do that ahead of time, with other system calls. This allows you to change the event list only when you need to, which is much less frequent than some data arrived from somewhere. The API is supposed to be O(1), instead of O(number of connections) per wait call.

My understanding is the kqueue works similarly, but I'm a Linux guy, so I can't really tell you.

select also has other problems w.r.t. FDs with high numbers.

doomrobo• 12 years ago

I think the only viable (in terms of portability) alternative is poll. There's a pretty good comparison of the two written by the author of cURL [0]. But essentially it's the same speed and doesn't have a hard-coded FD limit, but it runs on fewer platforms.

[0] http://daniel.haxx.se/docs/poll-vs-select.html

rdtsc5• 12 years ago

http://linux.die.net/man/7/socket

create a socket, bind, listen, connect, send etc.

Why do you need select?

rgbrenner• 12 years ago

this competes with libevent, libev, and libuv... all of which use the best method for the platform where it's installed.. so kqueue on BSD, epoll on Linux, etc.

That's one of the big reasons to use a lib for this.. so you get the best performance, without having to change your code to get it (or bother detecting which is best, etc).

haberman• 12 years ago

Recommended additions to the docs / marketing material:

"portable" -- portable to what?

comparisons -- vs. libevent, libuv, etc.

in the sample program -- when will dyad_getStreamCount() drop to zero? will it ever? what exactly is a "stream" anyway and how does it relate to individual connections?

codehero• 12 years ago

dyad_writef is certainly not 'async' when using the 'r' input specifier (which seems to redirect input from a FILE* to the output stream). And the way dyad_writef achieves 'async' is to buffer the entire contents of the output in RAM.

So for 'r', why bother using the FILE* formatted IO if all you're going to do is pull single characters at a time? You can just use file descriptors.

kbaker• 12 years ago

Here is a libevent echo server for comparison:

http://www.wangafu.net/~nickm/libevent-book/Ref8_listener.ht...

It looks like this library has only 2 weeks worth of commits so I'll reserve any criticism. My vote is still with libevent though!

larssonvomdach• 12 years ago

A friend of mine wrote something similar for C++: https://github.com/Kosta-Github/http-cpp

shroukkhan• 12 years ago

does not seem like it supports ssl socket..or did i miss it ?

eps• 12 years ago

There is no such thing as "ssl socket".

floatboth• 12 years ago

Is it called Dyad for the reason I think it is called Dyad? ;-)

dimman• 12 years ago

Is it just me that wants to have complete control over stuff? I mean if you really have the need to use a library like libevent you most likely know why and could do your own handling rather easily.

I'm not saying there's not a need for libraries like libevent, but IMO it's not needed for most applications. I might be biased because I want complete control and know what happens, I think that's important. I don't want to use a library before I know what happens in the "background". When I know that and know what I need, only then it might be appropriate to use a library.

matthavener• 12 years ago

I'm curious what you mean by "complete control"? Does that mean less abstraction? Or do you want to avoid inversion of control? That is, you want to avoid using callback-based libraries?

ishbits• 12 years ago

For me, libevent just provides the boiler plate but I'd be writing too often for fd registration and timeouts.

I could do it myself, but it would just end up looking more and more like libevent each time I did it.

Dyad: Minimal, portable async networking library for C

Related Stories