BlueHat IL 2023: Microsoft rewriting core Windows libraries in Rust

https://www.youtube.com/watch?v=8T6ClX-y2AE

91 44 3 years ago

bfrog• 3 years ago

Those memory bugs are costly after all. I think the best part of this presentation is they managed to improve performance while adding memory safety. Two people rebuilding a 90kloc c++ into rust in 6 months is also pretty impressive.

marcodiego• 3 years ago

Would it be a good idea to rewrite glibc in Rust?

bregma• 3 years ago

Rewrite the C standard library in Rust? I'm not sure I understand how that even makes sense. Are there problems this would solve?

mustache_kimono4• 3 years ago

A C standard library rewritten in Rust already exists. [0]

glibc has its share of CVEs, and many of the bugs are of the double free, use-after-free, buffer overflow variety. [1]

[0] https://github.com/redox-os/relibc [1]: https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=glibc

pjmlp5• 3 years ago

Many C standard libraries are actually re-written in C++ nowadays, e.g. Microsoft and clang's, so doing it in Rust is hardly any different, it would be safer than C++, minus the usual C gotchas that are anyway exposed on the public APIs and the implementations can't do much about it (like bad pointers or size).

toast0• 3 years ago

Rewrite the C standard library in Rust? I'm not sure I understand how that even makes sense. Are there problems this would solve?

The C standard library isn't necessarily a standard library of C code, it's more of a standard library for C code. There's plenty of languages these days where the standard library is built in other languages, somewhere between partially and fully; some portion of the C standard library can probably benefit greatly from the memory lifetime structures of Rust; although the constant need to interface with C code may reduce the effectiveness.

mustache_kimono4• 3 years ago

Already exists: https://github.com/redox-os/relibc

saagarjha3• 3 years ago

Can’t say whether it would be a good idea but it probably wouldn’t be a bad one.

curt15• 3 years ago

Since Rust doesn't yet support shared libraries, would each binary end up statically linking its own copy of glibc?

jcranmer• 3 years ago

#![crate_type = "cydlib"]

Rust supports shared libraries.

dmytrish• 3 years ago

Rust supports dynamic libraries with C ABI very well.

Technically, it also supports Rust ABI dynamic libraries, but their portability is quite fragile due to changes in the compiler. That's why Rust parts of the code are usually statically linked.

mrpippy• 3 years ago

Slides: https://msrndcdn360.blob.core.windows.net/bluehat/bluehatil/...

mustache_kimono4• 3 years ago

Rust discussion starts at: https://youtu.be/8T6ClX-y2AE?t=2610

Highlights:

    * 2 projects: dwrite and Win32k
    * 2 devs, 6 months to rewrite 152KLOC C++ to 90KLOC Rust re dwrite
    * Performance increases of 5-15% re dwrite
    * Only about 10% of Win32k code is unsafe
    * No perf regressions re Win32k
    * Syscall in the Windows kernel now written in Rust

zerr• 3 years ago

Is it really C++ or just C saved in .cpp files? (which is a norm in Win32 API world)

pjmlp5• 3 years ago

Only if the only thing one has read was Petzold's book.

Since MFC and COM, that outside of the kernel most code is C++, even MSVC libc is actually written in C++ with extern "C" declarations.

Additionally starting with Windows Vista, C++ code started to be allowed on the kernel (VC++ got /kernel flag), and nowadays there is a template library called WIL.

asveikau3• 3 years ago

Win32k is the kernel side of GDI, that's the core GUI drawing code for those unfamiliar with Windows.

It's a common source of privilege escalation bugs because it runs in kernel mode and accessible via a large surface of user mode apis.

pmarin3• 3 years ago

So code optimized for 80486 machines in the 90's is not less perfomace after the rewrite in Rust.

What a miracle!

Rust marketing is the best.

asveikau3• 3 years ago

You're getting downvoted but i think it's a fair point that revisiting win32k in any language might come up with some opportunities for optimization along the way. Performance best practices have changed a lot since that component was written.

camdenreslink• 3 years ago

Rust marketing says it is on par with performance with C/C++, not better. So I don’t really get your point.

biorach• 3 years ago

Rust marketing is the best.

This is not Rust marketing. This is some MS systems programmers detailing their experience with Rust.

hulitu• 3 years ago

That's because the bugs are also implemented in rust.

gabrielsroka• 3 years ago

The slide at 48m26s says 152 KLOC of Rust and 96 KLOC of C++.

I didn't watch the whole video though. Is that the same number you're quoting?

mustache_kimono4• 3 years ago

You are correct, and I was incorrect. It's actually the reverse of what I said, 152 KLOC of Rust compared to 96KLOC of C++.

muststopmyths• 3 years ago

Writing kernel-mode code in Windows with rust is very cool, but I am curious if there are code samples out there for this kind of thing.

Windows kernel drivers are usually fully asynchronous so I'm curious how rust handles IRPs and completion callbacks and such in the kernel.

steveklabnik3• 3 years ago

I haven't tried them out myself, but https://github.com/StephanvanSchaik/windows-kernel-rs exists. Has a bunch of code, though it hasn't been updated in a long time, so it may have bitrotted.

wolverine876• 3 years ago

Overlooked is that it's one of two incredible successes that have come out of Mozilla:

* The open web, David defeating Goliath Microsoft (arguably like a small open source retailer defeating Amazon today).

* Rust: Not entirely Mozilla, but they played an essential, major role nurturing it, and were among the first to deploy it.

Any others that I'm overlooking?

pjmlp5• 3 years ago

By the time of IE 5 came to be there was little left from David, hence why IE 6 was hardly having any updates, and most of the team went on to work on WPF.

kibwen• 3 years ago

Let's Encrypt was also originally from Mozilla.

In addition, I seem to recall that the Alliance For Open Media and its AV1 codec also derives from Mozilla sponsoring Xiph.org's Monty to come up with an unencumbered, next-gen media codec.

dralley• 3 years ago

Vorbis and later Opus contained a lot of contributions from Mozilla.

Buttons840• 3 years ago

One the subject of porting code from one language to another (ignoring that the headline says "rewrite", which implies more than a dumb port).

Are there any tools to help map out the work that needs to be done? I found myself doing this by hand recently, doing graph search on a piece of paper. I needed to port one function, so I identified all the functions that function called, wrote them down, and then looked into the next layer of functions etc. I manually searched the potential call graphs by reading the code and wrote down all possible functions involved. I then crossed of functions that seemed obvious, such as a function to retrieve the value of a dictionary with a default value, such simple functions are trivial and need no special effort. In the end I had a list of functions that needed to be ported one-by-one and I began working from the bottom up.

I was wondering if there are any tools that could automate this process? It seems to be the beginning of a source-to-source compiler, but such a compiler could be kept simple by embracing that most of the work can be presented to a human to perform manually. The "compilers" only job is to structure the work and break it down into small chunks.

Jtsummers• 3 years ago

https://en.wikipedia.org/wiki/Call_graph#Free_software_call_...

Lots of software tools for generating call graphs. I've used doxygen (free) for it in the past and LDRA (not free) at an early job. This won't generate the new code for you, but will be a lot easier than manually constructing a call graph.

rescbr• 3 years ago

I have used SourceTrail for that in the past, sadly its development stopped.

64operator• 3 years ago

MS and Amazon own and run Rust, this isn’t surprising.

saagarjha3• 3 years ago

They contribute to it; I don’t think it’s fair to say they own it? It’s certainly nothing like Swift or Go for example.

pjmlp5• 3 years ago

I guess they pay the salaries of key contributors, and certainly have some influence via their work.

BlueHat IL 2023: Microsoft rewriting core Windows libraries in Rust

Related Stories