The Rapid Growth of Io_uring

by signa11 • 6 years ago

213 65 6 years ago

willvarfar5• 6 years ago

I used to do asynchronous io with epoll, aio etc and spend time benchmarking my servers vs completion ports and kqueue etc.

eventually libraries like Libuv turned up and made my life a lot easier.

Are there any good stats on how io_uring compares to all those older async io stuff?

thedance• 6 years ago

Check the PDF that the article references, which also links to https://lore.kernel.org/linux-block/20190116175003.17880-1-a...

khc2• 6 years ago

epoll doesn't allow you to do async io to local storage, and io_uring doesn't target the network use case, so they are not quite comparable.

jabl• 6 years ago

Though it was originally designed for block io, IIRC io_uring has since added support for network sockets.

Don't know how it compares to epoll performance-wise.

topspin• 6 years ago

io_uring doesn't target the network use case

There are several io_uring opcodes intended specifically for the network use case, including: IORING_OP_ACCEPT, IORING_OP_CONNECT, IORING_OP_SENDMSG and IORING_OP_RECVMSG.

Did you mean something else by 'target'?

frevib• 6 years ago

Here are echo servers with

io_uring: https://github.com/frevib/io_uring-echo-server

epoll: https://github.com/frevib/epoll-echo-server

kqueue: https://github.com/frevib/kqueue-echo-server

io_uring performs much better than epoll: https://twitter.com/hielkedv/status/1218891982636027905?s=21

GrayShade2• 6 years ago

Not bad, according to one test: https://github.com/libuv/libuv/issues/1947#issuecomment-4852....

Note that it might have been optimized more in the meantime.

Matthias247• 6 years ago

That is for file IO, for which there were not many working async options available. I think the parent was rather interested in how it compares for IO loads where epoll & co are already alternatives.

mavam• 6 years ago

For users who need macOS and FreeBSD (kqueue) support as well, is there any unified standard for async I/O that covers both file and network? Or is the only choice to go with a library like libuv, which will always pick the optimal native implementation under the hood?

earenndil• 6 years ago

Once freebsd's linux implementation supports io_uring, you'll be able to use that there too. Nothing for macos, though, I'm afraid.

yxhuvud6• 6 years ago

Is there plans to support it?

cpach• 6 years ago

I’m afraid I don’t follow. What do you mean by “freebsd’s linux implementation”?

blattimwind3• 6 years ago

Or is the only choice to go with a library like libuv, which will always pick the optimal native implementation under the hood?

You mean thread-pools for file I/O?

willvarfar5• 6 years ago

So I started googling io_uring, and came across this excellent talk at FOSDEM https://fosdem.org/2020/schedule/event/rust_techniques_sled/

jen20• 6 years ago

This was indeed a great talk, and the Rio library it describes is also incredibly usable.

willvarfar5• 6 years ago

io_uring puts me in mind of a 2012 prototype of async and batched syscalls that sped up MySQL by 40%! https://www.usenix.org/legacy/events/osdi10/tech/full_papers...

The cost of syscalls has gone up massively due to all the mitigations against recent side channel attacks. Batching and async syscalls would be an even bigger win than ever.

Wouldn't it be great if glib or something could adopt new mechanisms and everyone got faster?

api• 6 years ago

In the long term I think OS APIs need to be redesigned (or at least the calling mechanism) around something more like io_uring for a lot of things. Shared memory, lock-free data structures, no syscalls.

willvarfar5• 6 years ago

And, back in 2012, cores were expensive. Now we have more cores than software to run on them, and having the kernel and user-space in respective busy nop loops waiting to get something is finally something to fill them up with?

The dynamics and mechanical sympathy of different approaches are changing as hardware goes massively parallel faster than software can.

tyingq4• 6 years ago

A popular standard cross process safe api to use mmap() for IPC, kv cache, locking, etc, might help as well. I understand it's already possible, but it's either reinvented every time, or eschewed for cross-host...often when not needed.

cesarb3• 6 years ago

Shared memory, lock-free data structures, no syscalls.

You'd still need a syscall to sleep when the ring is empty, and a syscall to wake up the mechanism when the first request is put on an empty ring. But yeah, other than that (and perhaps an "yield" syscall), you could do everything through the ring.

BubRoss2• 6 years ago

Reinventing QNX will be cutting edge for decades

hinkley2• 6 years ago

I’d be curious to see some statistics about the number of system calls per second now versus ten years ago. I have a suspicion they’re much higher.

If the scale of the problem changes, so does the solution. We know this.

executesorder66• 6 years ago

Is there any userspace software that is actually making use of io_uring?

saghul• 6 years ago

There is an ongoing effort to bring it into libuv: https://github.com/libuv/libuv/pull/2322 Looks like using liburing might be the way to go, since it's MIT licensed.

PudgePacket• 6 years ago

https://github.com/spacejam/sled via https://github.com/spacejam/rio

birkelund• 6 years ago

QEMU just merged io_uring support as an aio engine for block I/O.

cytzol• 6 years ago

Question: is io_uring only worth using for writing to files, or does it provide performance benefits when reading from them, as well?

I've seen database programmers rave about the speed benefits of asynchronous I/O, because they have to store a lot of data on disk. But the majority of programs I write only have to deal with reading files. I'd love to try using io_uring, but only when it's appropriate.

thristian• 6 years ago

Asynchronous I/O is only useful if you have something else you can be doing while you wait for file-data to arrive. For example, if you can do calculations on the part of the file you've already read while you wait for the rest to arrive, or if you can process data for one client while data from a different client is in-flight. Databases are a good example - usually they're handling queries for multiple clients at once, and even within a query they can be reading from multiple places simultaneously (like the individual tables involved in a JOIN).

If your program needs to read in an entire JSON blob (for example) before it can do anything, or if it only does light processing on each individual part (like adding up a column of a CSV file) then async I/O probably isn't going to help.

machinecoffee• 6 years ago

Which would actually work nicely with a fibres library, so you can yield while you're waiting for the io to complete.

yxhuvud6• 6 years ago

You can for example read from multiple files simultaneously.

You can also build web servers that read from disk as part of the request handling, without having to do blocking file reads. Before io_uring, this scenario would get absolutely horrible performance as the whole thread would lock down while waiting for the file read. Now you can do other things concurrently.

cmsd2• 6 years ago

it looks like in the future it might be a better way of implementing khttpd by handing over from a userspace control program the accept->read->sendfile->close sequence of async operations to the kernel

willvarfar5• 6 years ago

Imagine a database trying to solve

    SELECT client_id, invoice_id, invoice_status, invoice_date
    FROM invoices
    WHERE client_id IN (123, 567) AND invoice_status != 'PAID';

Imagine that there is an index for client_id.

The DB engine can scan the index, finding all the pages in the table that contain rows with one of those two client_ids in.

It requests these pages as soon as it finds them in the index.

It scans the pages in the order it finds them completed, though, which can be a completely different order than it requested them in.

blattimwind3• 6 years ago

Turns out, if you give people a sane API, they actually want to use it! And this is the first relatively sane way on Linux/BSD to do asynchronous IO. :)

(I say "relatively sane" because this isn't really asynchronous, it's just make-believe with a kernel-managed thread pool, because I/O being a fully synchronous affair is ingrained far too deep into both Linux and BSD I/O stacks.)

wtallis2• 6 years ago

"relatively sane" because this isn't really asynchronous

What's your criteria for "really asynchronous"?

it's just make-believe with a kernel-managed thread pool

To the extent that io_uring uses anything resembling a thread pool, it seems to me that it is used completely differently from how a userspace AIO thread pool operates. When a userspace AIO implementation submits IO to the kernel, it does so with a blocking syscall and that thread stalls until that IO is complete. That means the number of outstanding IOs is limited by the number of threads in the pool. I don't see any such limitation in using io_uring to deliver IO requests to the block layer.

tyingq4• 6 years ago

What's your criteria for "really asynchronous"

One example might be that related operations, like stat(), opendir(), readdir(), getpeername(), and so on...remain synchronous. And that async functionality is mostly a bolt-on to very established things, file descriptors, berkeley sockets etc.

Also, every improvement is a pretty big patchset with code to ensure the traditional synchronous operations don't get unintended side effects.

Just generally the idea that a "clean start", non-POSIX bound OS might design things differently, I imagine. Google's Fuchsia seems to hit some middle ground, where async is more foundational, for example.

I don't think that observation detracts from the improvements.

blattimwind3• 6 years ago

I don't see any such limitation in using io_uring to deliver IO requests to the block layer.

You can already have asynchronous IO from userspace to the block layer via linux-aio and O_DIRECT. But the VFS remains synchronous, so both uring and userspace can effectively only use a thread pool to work around that. Or magic.

mzs• 6 years ago

Actually this is FB wants something bad enough and Jens Axboe works for them.

rayiner2• 6 years ago

One of the comments to the article is really interesting. The recent Meltdown stuff has really blown up the cost of privilege transitions, because now people expect non-architectural data not to leak across privilege boundaries. System calls are about twice as slow as they used to be. Meanwhile, I/O is faster than ever, with PCIE and NVME. Io_uring offers the opportunity to avoid privilege transitions through asynchronous calls based on writing to a memory buffer shared between user space and the kernel. That has the potential for fundamentally changing the basis system call interface. As the article hints at, the trick is designing the API so you can construct as large a block of work to be done asynchronously as possible. At the limit, you could really push down the core of your I/O loop into the kernel, hence the suggestions that BPF programs could be submitted through the ring to chain operations all within the kernel.

(Incidentally, this is a good illustration of the flexibility of UNIX’s model of describing everything with a file descriptor. The same interface meant for asynchronous file I/O was easily extended to network I/O.)

stefan_• 6 years ago

hence the suggestions that BPF programs could be submitted through the ring to chain operations all within the kernel

That sounds unbelievably annoying.

No, the limit has been found and it is not in the kernel. You push the I/O loop up for userspace I/O, more specific, not more general. They call it SPDK [1] (or DPDK for networking) and as far I can tell, the principle is essentially having a dummy driver in the kernel that maps the entire PCIe peripheral memory space into your chosen process, and everything flows from there.

At the I/O limit, asynchronous isn't feasible because interrupts introduce latency and waste cycles not doing work. All userspace I/O frameworks work only through polling.

1: https://spdk.io/

couchand2• 6 years ago

At the I/O limit, asynchronous isn't feasible because interrupts introduce latency and waste cycles not doing work.

io_uring supports polled i/o: https://lore.kernel.org/linux-block/20190116175003.17880-8-a...

wtallis2• 6 years ago

SPDK is a pain to use. io_uring supports polling, and that gets it within a few percent of SPDK performance.

rayiner2• 6 years ago

The problem with mechanisms like DPDK is that they bypass all the infrastructure in the kernel and make it hard to play well with others using the same hardware or services. DPDK, for example, bypasses the TCP/IP stack. SPDK bypasses the VFS. You can write your own TCP/IP stack or filesystem on top of those things, but then you can't play well with other processes using those services. While some GPUs can directly multiplex command streams from different processes, most hardware cannot.

MayeulC• 6 years ago

Interestingly, the advent of these asynchronous, context-switch-less interfaces might see microkernels make a comeback, as they were originally decried for performance reasons (they traditionally need many expensive context switches to operate).

hinkley2• 6 years ago

I wonder too if in this batched model if there's some opportunity to further leverage the 'race to idle' solution you see in mobile devices. If I have 16 cores and half of them are frequently spun down, the thermal budget for the others is improved.

matheusmoreira• 6 years ago

Linux finally has something comparable to Windows I/O Completion Ports. Looks like it is flexible enough to be a generic asynchronous system call interface. Awesome!

kashyapc• 6 years ago

From last sunday, at FOSDEM 2020, a talk[1] by QEMU developer Julia Suvorova, on integrating io_uring into QEMU (the open source machine emulator and virtualizer), and related performance implications.

From the abstract:

"iouring enhances the existing Linux AIO API, and provides QEMU a flexible interface, allowing you to use the desired set of features: submission polling, completion polling, fd and memory buffer registration. By explaining these features we will come to examples of how and when you need to use them to get the most out of iouring. Expect many benchmarks with different QEMU I/O engines and userspace storage solutions (SPDK).

You will get a brief overview of the new kernel feature, how we used it in QEMU, combined its capabilities to speed up storage in VMs and what performance we achieved. Should io_uring be the new default AIO engine in QEMU? Come and find out!"

[1] io_uring in QEMU: high-performance disk I/O for Linux — https://fosdem.org/2020/schedule/event/vai_io_uring_in_qemu/

The Rapid Growth of Io_uring

Related Stories