Show HN: Libriscv – RISC-V userspace emulator library

by fwsgonzo • 4 years ago

40 8 4 years ago

fwsgonzo4• 4 years ago

Maybe a few months from now it may be able to run Linux as well. I had painted myself into a corner by processing execute segments in order to get more speed. However, I have currently found ways to have many execute segments, and before that I solved how to have memory traps without undue memory bandwidth pessimization. So, I have some hope that I may be able to support the Supervisor extension in the coming months, as all the puzzle pieces seems to be coming together. The emulator has had a fully virtual address space from the start, so the door has always been a little bit open.

I don't know if my plan is foolproof, but the plan is to trap on every page in page tables and if any changes are made, possibly followed by some kind of page table invalidation, I will be able to reconstruct the execute segments in full. In full is key here, because compressed instructions are impossible to seek backwards on, so for changes to happen I will need to find the first execute page, and then build from there.

At any rate, it is also possible to forego the faster instruction dispatch while I am building support for the Supervisor extension, which will greatly aid sanity.

I'm pretty happy with the emulator these days as it is still fairly thin, and hardly uses any memory to run programs. It is also running Go programs, with exception to the HTTP client example. I don't know what's keeping it from working, but I suppose I will eventually figure it out!

klelatti2• 4 years ago

This looks like a fascinating project that you've all put a huge amount of work into.

Can I ask more about the motivations / use cases you see for it? You hint at games engines but I'm not clear how this would work in practice and what any others might be. Thanks!

fwsgonzo4• 4 years ago

There are many places it would be OK to use it. Anywhere where you have extreme requirements on low latency, which I like to call ultra-low-latency, like a high-performance daemon. You can fork this emulator from another and execute the first instruction in less than 1 microsecond. In those environments the most important factors are: How many can you have? How many Linux system calls do you need? What is the cost of making an API call (trap out of the emulator)? The answer to these questions is that you can have hundreds of thousands of emulators at the same time, because forking just means loaning data from a main emulator, they make no Linux system calls, in fact they don't even use mmap at all. And, trapping out to handle a system call has the same time-cost as calling an opaque function.

This emulator will never have the same near-native speed as WebAssembly because WASM uses a linear arena for "heap". Memory checks can be as simple as one comparison, sometimes even none at all. However, that arena is allocated, extended and cleared using mmap-family of system calls, each of which may require page-table locking. WASM also uses JIT, which I personally think is large attack surface. If you search on HN you can find engineers of SpiderMonkey and v8 saying the JIT is just completely out of control in terms of complexity. This is the reason why I was experimenting with binary translation using C and GCC. I think it turned out OK, and the complexity ended up being fairly low. I actually beat native Go fibonacci using the code.

TLDR: This is my attempt at doing something different from others, and it turned out to be good at some obscure things, while being bad at others.

Other than that, it's just a hobby project. The main reason I am working on it still has been that it's my most fun project. It's just very cool to be able to sandbox complicated programs. On that note I was actually trying to run Doom 1 (shareware) in my emulator yesterday, and I think I was rendering too many frames per second in SDL2 because my Lenovo Z13 completely blacked out and when it came back online the network card didn't show up. Not because the emulator is so blazing fast, probably just because of lack of vsync. Had me scared for a while, but I discharged the machine and now the device is showing up again :S

YorkshireSeason• 4 years ago

Nice work!

How did you debug it? Did you run it against the 'official' RISC-V models Spike [1] or Sail [2]?

[1] https://github.com/riscv-software-src/riscv-isa-sim

[2] https://github.com/riscv/sail-riscv

fwsgonzo4• 4 years ago

Yes, I used Spike quite a bit in the beginning. I also compared with my own instruction logging with objdump -drl program | less.

These days I have a decent test suite. I verify Go and Zig hello worlds (both running through a surprising amount of code): https://github.com/fwsGonzo/libriscv/blob/master/tests/unit/...

And I generate a mandelbrot that I compare: https://github.com/fwsGonzo/libriscv/blob/master/tests/unit/...

While most tests produce RISC-V programs which are run to completion as part of the test, that still might not cover every edge case, though. I have tried to integrate with the riscv-tests repository, but it uses a lot of privileged instructions. After I have implemented more of the supervisor extension I will be able to run those tests.

sylware• 4 years ago

Mr Bellard did that too: https://bellard.org

fwsgonzo4• 4 years ago

There is a doom emulation demo here now: https://github.com/fwsGonzo/libriscv/tree/master/emulator/do...

You will need to add the shareware doom1.wad yourself. :)

Show HN: Libriscv – RISC-V userspace emulator library

Related Stories