HN user

kqueue

265 karma
Posts8
Comments195
View on HN
Announcing Amber.js 15 years ago

I tried using SC2 (aka Amber), while it has a lot of potential and I love the powerful binding mechanism it has, it simply lacked documentation and support. I hope Amber has tackled this issue.

Gmail’s new look 15 years ago

I don't like fixed position UI and scrolling within a small section. I prefer the whole page to scroll.

I am staying on the old version.

I totally agree that SYSCALL/SYSENTER/SYSRET are very cheap to execute. But these instructions only takes care of the ring switch and are not executed alone.

When you make a system call, a trap is issued that causes the hardware switch to kernel mode.

The hardware pushes onto the per-process kernel stack the pc, status word, and the kernel code takes care of saving the registers, esp, etc.. this is called a task context switch. Context switching between processes is much more expensive but task context switching is still considered a context switch.

When you are making a system call, it's still much more expensive then most of the work you are doing in your program hellepoll and hence it's your bottleneck. This is why you don't see your process's CPU at 100%.

On a related note, whenever you have a program doing a lot of network IO, you are essentially causing a lot of process context switches because each time you get data on the wire you cause a context switch because the kernel needs to handle this hardware interrupt.

I didn't say there is no distinction. I said system calls that require disk io, or network io, or triggers locking/sleeping requires a context switch. Calls like read/write/accept triggers context switches (which starts with a mode switch). The switch is required for the kernel to execute the system calls and operate on it's own data structures. Only the kernel can alter mbufs in this case.

Go read your OS book.

I am not trolling. you just lack experience and this sounds new to you.

Network IO, Disk IO, scheduling, locks etc.. all trigger context switches and not only mode switch because only the kernel is allowed to manipulate data structures related to mbufs, vfs, and whatnot.

When I said epoll/kqueue doesn't affect the accept rate, I was replying back to a specific request of writing a program to just accept/reply/shutdown. In this case you are passing one fd to the poller which won't matter much what poller you are using.

My original comment is that you are measuring the wrong thing and it's not against hellepoll. requests / sec can be much higher than reply / sec because they get buffered waiting for you to accept them. Once accepted, then a request is counted. What matters though, especially to the http client is how long it takes to serve the connection from start to finish, hence reply rate.

fyi, you can use libev for poller portability.

You can experiment with this command and look at the reply rate.

httperf --num-conns=10000000 -vv --num-calls=1 --port=<your_port>

request rate is not influenced much by the polling mechanism (epoll/kqueue/poll/select) when all you're doing is listening to one fd and processing new ones then closing then immediately. These multiplexers matter when you are working on lots of file descriptors.

It matters because select will have to iterate over all the file descriptors you passed to it, while kqueue for example have knotes registered to it when it wakes up and won't need to iterate over everything, but just the knotes it got. Not to mention the data copying from userland to the kernel in case of select/poll.

Back to the request rate limitations, the limitation is mainly coming from the # of system calls you can execute in a sec (accept() in this case) which is heavily influenced by the context switches.

Looking at the application CPU gives you a very good idea of what your bottleneck is. If your CPU is 100% then it's clear your application is hitting the limit, but if the application CPU is at 30% and you cannot process more requests / sec then you've hit the system limit here.

I believe you are testing the requests / sec and not replies / sec. While requests / sec matters a bit, you'll most probably be bottle-necking in your context switches. What matters is replies / sec. This is a more accurate measurement of your server performance and not OS bottlenecks. I'd use httperf for this matter.

I see. I haven't tried them personally, my friend has been using them for a year now and so far so good. He did complain about their support though.

Not to mention that kevent contain the number of bytes to be read, or the number of connections to accept, which can avoid you a system call that returns EAGAIN in a non-blocking socket.

1. Process creation and termination are heavy operations considering that it's being done in a tight loop.

2. You don't want to fork on every request, this is very vulnerable to fork bombs.

3. In worker threads model, you already have the work threads spawned and ready for crunching, which will reduce the system load because you aren't forking on every request.

You cannot render a pattern useless because you had a bad experience with certain implementations.

The pattern represent a very good abstraction concept that is irrelevant to the library implemtation.

> Intruders gained root access on the server Hera. We believe they may have gained this access via a compromised user credential; how they managed to exploit that to root access is currently unknown and is being investigated.

This is more related to weak security policies than OS security flaw.

Show HN: groups.io 15 years ago

I have internal metrics. Nothing too complicated just few measurements I care about.

Thanks