How Not to Write a Signal Handler

by nkurz • 12 years ago

141 49 12 years ago

majke• 12 years ago

I still don't understand how queueing signals works.

For example, signalfd() is really cool, and you can indeed read() a signal from it, but only one of a kind.

If you get, say, two SIGINT's you may as well just be able to read() one. I guess it's an implementation detail that some signals aren't queued while some others are, I guess. In practice it means that receiving a signal is an edge, and in signal handler you must check how many times the signal occurred. This for me sounds inherently racey.

caf• 12 years ago

You don't check how many times the signal occured - you write your code so that it doesn't matter how many times the signal occured.

For example, when you're handling a SIGCHLD you don't assume exactly one child has exited - you loop around calling waitpid() with WNOHANG and handle every child that's exited.

general_failure• 12 years ago

What's a good use case for knowing how many times a signal was raised? Signal once raised has no more context than the signal itself.

rurounijones• 12 years ago

If the signal is received more than once it means it is more important :p

"Oh shit!"

"^C^C^C^C^C^C~C^C"

anuragbiyani• 12 years ago

Although I cannot recall any specific name right now, I have certainly seen programs handling a single Ctrl-C different than say two Ctrl-C in quick succession.

Example: Pressing one Ctrl-C will display a message like this on some programs (which do not want to terminate themselves upon receiving a single SIGINT, because accidents happen): "Press Ctrl-C again to quit."

noselasd• 12 years ago

Note that the standard signals are not queued, a signal handler for a specific signal might be run just once if several signals are sent between the first signal arriving and the signal being handled.

As an extension, there are "real-time" signals as well, which are queued - linux implement them, see e.g. http://man7.org/linux/man-pages/man7/signal.7.html

vezzy-fnord• 12 years ago

I think it's a shame (and painful) how the major Unices have all significantly diverged on things like event handling, I/O multiplexing and file system watching.

I find the BSD kqueue(2) interface to be much more elegant than any of Linux's *fd() functions or epoll(7). On the other hand, Linux's inotify(7) is, I find, cleaner to use for file system event notifications than kqueue(2). But then there's also fanotify(7) on Linux, which seems to have sort of been neglected since its initial hype. Then OS X has FSEvents.

I guess this is why one may need libraries like libevent and libuv.

fafner5• 12 years ago

I think the kqueue interface is a lot worse than the Linux way. With kqueue everything is forced through one interface. In Linux it's just file descriptors and epoll to deal with them. It is a nice extension of the "everything is a file" mechanism and it embodies the "do one thing and do it right".

This becomes apparent when you look at fs notifications. You've already mentioned how inotify is superior to kqueue. I mean seriously, the kqueue fs notifications are designed in such an incredibly bad way, it almost looks like a trolling attempt. E.g., when you try to watch a directory the kernel knows which files are manipulated. But the API has no way of telling you. The documentations usually suggest to keep a file list and update it. But this is complicated and racy and will certainly result in buggy behaviour.

And that really shows that having one interface do everything is a bad approach. The kqueue designers had to cover every event type that could happen. And thus they added an API for something they apparently didn't understand properly. And since it's stuck now in their API they would have to deprecate parts of it, if they ever have the interest in fixing it.

The inotify API is not perfect. But it does its job pretty well and it is the best fs notification API I have seen. fanotify solves a different use case. And that's really the flexibility that the Linux interface has. They can easily come up with new event APIs for new use cases because all they have to do is provide a file descriptor.

I wish the BSD folks would simply implement inotify. But it seems they are unwilling to fix their API and if they did then they'd probably design their own interface simply out of spite. Right now their crappy fs notification interface is a real pain. And since the lowest common denominator is very influential when it comes to portability libraries like glib, Qt, etc. everybody is suffering because of BSD. And they only get away with it because OSX has the same API. I haven't looked at FSEvents. Is it a new API or simply based on kqueue?

</rant>

kev009• 12 years ago

There's been talk (at BSDCam) of implementing inotify which is needed for the Linuxulator and exposing it in the FreeBSD API too. So your wish isn't so far fetched but I'm not sure anyone is actively working on it. Patches welcome :)

teacup508• 12 years ago

Bah. This is complete nonsense.

EVFILT_VNODE is not complicated to use for the cases for which it was actually designed — monitoring a file descriptor. For more general file system monitoring, a new kqueue filter type would be ideal, but adding that does not require breaking or deprecating kqueue API; the whole point of kqueue is to provide a generic extensible event mechanism.

Adding a recursive path monitoring filter is as simple as:

- Defining a new `EVFILT_DIR` filter type.

- Accepting a file path or st_dev/ino_t pair via the generic `uinptr_t ident` kevent identifier.

- Providing extended data via the existing filter-controlled `intptr_t data` value.

Boom. Done. Using the nice, generic, well-designed kqueue() API that can also monitor file descriptors, processes, AIO events, signals, timers, and user-defined events.

I wish Linux — like Mac OS X, and all the BSDs — had adopted kqueue, or at least participated in the conversation. Instead, Linux went through 2-3 different mechanisms before finally settling on the odd-duck single-purpose inotify interface, despite kqueue's design having been published (http://people.freebsd.org/~jlemon/papers/kqueue.pdf) and released as part of FreeBSD 5 years earlier.

While Jon Lemon published a detailed paper covering kqueue's design, implementation, and performance benchmarking, the inotify developer published a 30 line README with erudite gems such as "Rumor is that the "d" in "dnotify" does not stand for "directory" but for "suck.": https://www.kernel.org/pub/linux/kernel/people/rml/inotify/R...

lkrubner• 12 years ago

They write:

"It is the 2013th year in the Common Era at the moment of this writing and you might think that people should have came up with something better in terms of signal handling at this time. The truth is that they did. It is just not that well known yet due to a huge momentum of outdated information still overflowing the Internet."

Then they talk about kqueue.

The above paragraph emphasizes how much Unix programming still relies on old ideas.

Likewise, sockets are an old idea, and newer libraries, such as ZeroMQ, do a lot to fix the old problems. ZeroMQ is often described as "Sockets on steroids". It implements a lot of patterns that Unix itself does not give us:

"It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fan-out, pub-sub, task distribution, and request-reply. "

But I am left wondering, what would the world of programming be like if we had a new operating system that incorporated some of the new ideas and patterns that have developed over the last 25 years? Instead of depending on libraries, would we not be better off if we had an OS built around these newer ideas?

sillysaurus32• 12 years ago

We wouldn't be better off, no. The newer OS would likely be less secure, less reliable, and probably less performant. It would also require all of the tools we take for granted to be ported to the new OS, which would be difficult since the above proposal is to change the API of the new OS.

It doesn't seem like there's anything wrong with libraries.

wtetzner• 12 years ago

I don't see why it would be less secure, reliable, or performant. It might be that the new ideas, and new abstractions, make it harder to do things in insecure or unreliable ways.

angersock• 12 years ago

Have you heard of...

...Erlang? :)

Most of the real problems were solved years ago--they just haven't been used because reasons. :(

teacup508• 12 years ago

I don't understand your point; kqueue is a comparatively new idea.

Do you want signals written as messages across ZeroMQ? In that case, look at Mac OS X's use of Mach ports for exception handling, which is an even older idea than kqueue.

lmm• 12 years ago

It's been tried. Look at plan 9, or MS Singularity, or heck even BeOS was a clean break in many respects, with certain things implemented at low level and used throughout the OS.

Turns out compatibility and understandability is really useful, and it's good to define your complex systems in terms of simple systems. Worse is better.

julie1• 12 years ago

why not talk about soft IRQ?

The signal idea is to implement a user space soft IRQ like communication between process.

But we trigger them the wrong way

if should not be syslog that does kill -HUP daemon to have them release file but daemon that should have callback implemented like "on_file_destroy" and this event should be propagated to the root processes until the bubble up event is being stopped explicitly from being propagated.

The problem with that approach that is simple is it requires a careful definition of defined callbacks

digisign• 12 years ago

I looked around but couldn't find a great answer on how to do this correctly in Python on Linux. I have a daemon that attempts to shut down cleanly when it gets a number of signals and it appears to be working correctly. But, it's written like the first example.

whopa• 12 years ago

Python does the right thing under the hood for you. The signal handler you register in Python isn't run in the dangerous context. There's an internal signal handler which just tells the interpreter a signal happened and when control is returned back to the interpreter outside of the signal handler it knows what to do:

https://hg.python.org/cpython/file/c7d45da654ee/Modules/sign...

maxlybbert• 12 years ago

It is possible to write thread safe signal (and portable) signal handlers doing exactly what you've done. Have the handler set a flag. That's is. Then, outside the handler, periodically check that flag and do the real work when it changes.

It used to be that you needed to make the flag variable "volatile sig_atomic_t" ( http://pubs.opengroup.org/onlinepubs/009695399/basedefs/sign... ). I think nowadays you'd want to use an atomic bool or atomic int.

The main reason I prefer this approach is that sigaction is in the POSIX standard but not the C or C++ standards. sigaction isn't available on Windows, but signal is (then again, very few signals are available on Windows).

pmorici• 12 years ago

Is there a reason they are using sleep(1) instead of pause()? Seems like a waste to use sleep if all you are doing is waiting for a signal.

spoiler• 12 years ago

Well, you can's make a funky comment about `pause` being essential, and `sleep` is essential.

lelf• 12 years ago

you might think that people should have came up with something better in terms of signal handling at this time. The truth is that they did

… and then the article goes completely omitting sigqueue+sigwaitinfo (realtime signals)

userbinator• 12 years ago

I've always found it interesting that the list of safe functions includes read(), write(), and most of the filesystem operations; so while printf is not safe, you could use write() if you really wanted to output.

gryph0n• 12 years ago

write() is an OS system call, while printf() is a C function call.

C libraries can (and do) choose to buffer printf, while read/write are usually unbuffered.

Of course, both are mechanisms are modifying some global state (your terminal's state), and cannot be relied to be free of race-conditions.

achivetta• 12 years ago

As far as "modern signal handling" goes: On OS X, you can also just create a dispatch source of type DISPATCH_SOURCE_TYPE_SIGNAL. If your code already uses dispatch, this is much easier than trying to wire up to kqueue directly.

qznc• 12 years ago

Why does the signal handler have to interrupt a thread? Why doesn't the OS just create a new thread to run the signal handler in? This would avoid this whole reentrancy problem and make the concurrency explicit.

teacup508• 12 years ago

The OS has to pause a thread that triggers a signal; how can it proceed if doing so would just segfault again?

fulafel• 12 years ago

Only a subset of signals come from mapping CPU faults to signals, the so-called synchronous signals. This example was about SIGINT coming from the usual source (Ctrl-C from terminal).

It's actually possible to handle it in a thread using existing semantics without kqueue or signalfd: block SIGINT from all threads except your dedicated SIGINT-handling thread.

Athas• 12 years ago

Threads are fairly expensive - you have to allocate and map a new stack, as well as a bunch of bookkeeping data structures in the kernel. While the kernel could of course start an auxillary thread at program initialisation, and re-use it for all signals, this does complicate the runtime environment a bit. And what happens if the signal-handling-thread triggers a signal?

I suppose the Unix tradition encourages giving the user the ability to centralise signal handling in a thread if he wishes (using kpoll or signalfd or whatever you want), but not forcing any overhead upon the program.

jheriko• 12 years ago

this seems like yet another case where a thread safe lockless queue would be good. setting a volatile atomically is fine for the simplest cases, but what if you want to handle every signal raised independently and not just do something if any signal is raised?

the real answer here i think is to think about code being run concurrently and how to do that safely... where a signal handler is a special case that, actually, requires no extra special treatment.

How Not to Write a Signal Handler

Related Stories