Digging into a QEMU problem of slow data copying

https://linus.schreibt.jetzt/posts/qemu-9p-performance.html

by todsacerdoti • 4 years ago

190 31 4 years ago

linus.schreibt.jetzt

tracnar2• 4 years ago

Nice fix. I wonder if using virtiofs would be more performant: https://virtio-fs.gitlab.io/

I've been planning to implement NixOS tests based on virtiofs instead of 9p for a while, but haven't got it working yet (at the time I tried, the available uid mapping strategies didn't cover what was needed for NixOS tests; there's a patch which I haven't got around to trying out yet though.) and this big improvement takes the pain away a bit so my motivation to finish that off is lower. I'll be sure to write about it if I do though!

tracnar2• 4 years ago

Great, curious to see what you discover!

rwmj• 4 years ago

All the development attention is on virtiofs these days and 9pfs is basically deprecated. QEMU developers would certainly recommend switching.

tinus_hn• 4 years ago

Doctors don’t want you to know this one trick to make me 10 times less likely to avoid your article: don’t use these cringey clickbait titles.

dang• 4 years ago

It's clearly a good article otherwise. We've changed the title now—if there's a more representative phrase from the article itself, we can change it again.

Submitters: "Please use the original title, unless it is misleading or linkbait;" - this one was linkbait, so should not have been used. https://news.ycombinator.com/newsguidelines.html

ozzy6009• 4 years ago

This is awesome, thanks for sharing so much detail. Btw one thing I noticed, the link to "Determinate Systems" was broken. I think this is the correct link: https://determinate.systems/posts/qemu-fix

linuxhackerman3• 4 years ago

Great job, me. I fixed it in the source but didn't deploy it to the site. Thanks for pointing it out!

lithiumii2• 4 years ago

I wonder if this will fix the slow IO in WSL2 reading from Windows (/mnt/c), since WSL2 also uses 9p

spijdar• 4 years ago

No, since WSL2 is based on the Windows hypervisor/emulator Hyper-V, not QEMU/KVM.

lithiumii2• 4 years ago

But when you access files on Windows from WSL2 (e.g., under /mnt/c), the same 9P protocol is being used

lathiat• 4 years ago

Right but this fix was on the “qemu 9p server”.

On WSL you would be using the “hyperv 9p server”.

Same Linux 9p client in both cases but that’s not where the fix was.

Reasonable question but the answer is still no.

idiocrat• 4 years ago

From commit: > The previous implementation would iterate over the fid table for lookup operations, resulting in an operation with O(n) complexity on the number of open files and poor cache locality -- for every open, stat, read, write, etc operation.

Windows's file handling also is very slow at times (especially Windows Explorer has frustrating UX).

We can ask MS to release all their source and we will then fix all O(n) issues for them.

zx2c4• 4 years ago

The relevant commit is here: https://github.com/cschoenebeck/qemu/commit/8ab70b8958a8f9cb...

anderspitman2• 4 years ago

I wonder if this is using slirp networking or a TAP device.

linuxhackerman3• 4 years ago

It doesn't actually go through a "real network" at all, the VM speaks 9p to the host via a virtio pipe (not sure what the technically correct term for this is) so there's none of the overhead that you'd have with emulating a network device and encapsulating it all in Ethernet/IP/TCP.

anderspitman2• 4 years ago

Ah right I forgot QEMU has built in 9p on Linux. Sadly it's not yet available for Windows hosts, though there's a patch in the works.

You have any suggestions for getting a foothold in the QEMU codebase?

pabs3• 4 years ago

For those who haven't used patches via email before and want to contribute to Linux/qemu/etc that use that, there is a neat site about how to set it up and do it:

https://git-send-email.io/

voidwtf• 4 years ago

The idea of using emails for patch management fills me with abject horror.

To each their own, I just don’t understand why anyone would prefer this method.

nezirus• 4 years ago

Nice story! It also matches my experience with many open source projects. It is not enough to solve a technical problem, you must convince other people to accept the solution.

Having a nice maintainer to guide you through the process is really encouraging and brings more nice and bright people to the comunity. Kudos

stabbles2• 4 years ago

I opted for a hash table, as conveniently implemented by glib, which QEMU already depends on

I wonder how many C projects use linear scans instead of hash maps simply because C doesn't have it in its standard library.

tubler• 4 years ago

I find it odd how all the innovations in getting broadly useful algorithms into a standard library happened in C++ but not C.

It almost feels like the people who keep churning out new standards of C just gave up and decided that part of it couldn't be improved any further.

junon2• 4 years ago

Eh. No. C is meant to be lightweight. Many people, including pro-C++ individuals, agree the C++ standard library is way beyond bloated.

stabbles2• 4 years ago

My guess: in C++ it's templated and header-only, which means optimized hash table code gets generated for the types you use it for, and you don't pay for it in binary size when you don't use it. In C you get stuff like qsort which is a single symbol in the libc binary, to which you have to pass a comparison function as a pointer, which is awkward and slow because it isn't inlined (unless you do link-time optimization, do people really?).

junon2• 4 years ago

I've certainly run into this a non-trivial amount of times myself. It's the one thing I'm really missing in C.

MontyCarloHall• 4 years ago

I frankly don't understand why projects [*] are still written in pure C. With the exception of a few edge cases, C is pretty much a subset of C++, so you can easily write code that's 100% C, except for when useful data structures from the STL are required. In this specific case, what is the disadvantage of writing C code that pulls in std::unordered_map when a hash table is needed, but otherwise does not use any other C++ language features?

[*] aside from those targeting niche, embedded systems for which C++ is unavailable

199955347• 4 years ago

Sandi game

astro1• 4 years ago

FWIW, the contemporary alternative to 9p is virtiofs. It is implemented as a separate service that is running aside the hypervisor. Originally out of qemu, it was rewritten in Rust as a freestanding project that can also be used with cloud-hypervisor and crosvm.

https://gitlab.com/virtio-fs/virtiofsd

On NixOS: https://github.com/astro/microvm.nix

Digging into a QEMU problem of slow data copying

Related Stories