First Rust code in the Windows 11 kernel

https://www.thurrott.com/windows/windows-11/282995/first-rust-code-shows-up-in-the-windows-11-kernel

by wojtczyk • 3 years ago

206 165 3 years ago

solarkraft• 3 years ago

Damn! If I'm correctly informed, this is before Linux is doing it (actually shipping rust code, not having the infrastructure in place).

therein• 3 years ago

I feel like there is a way more vibrant driver developer community for Windows anyway. It could be the architecture for kernel drivers Windows has or that many people use it for malware and game cheat development. I have a fond attachment to all the tools you have in your disposal once loaded into the kernel space on Windows. win32kbase.sys being at a certain memory location for all processes, KeUserModeCallback etc. all around really fun and interesting to hack with. Doing DKOM on PEB, TEB of kernel processes or user processes to do things you shouldn't be able to do like call win32 apis from kernel pretending to be a process to draw things using GDI etc.

Even though I primarily am a Linux user and I prefer using Linux on a daily basis, dabbling in Linux kernel development failed to create this feeling of awe and infinite possibilities (and exploitability) in my mind.

I guess my point is, Windows Kernel development in Rust excites me. Same with Linux would be very cool but it doesn't create the same feeling of wonder in me. Perhaps that's why this is happening first.

dilawar• 3 years ago

https://github.com/SubconsciousCompute/poc-windows-rust-filt...

We have been playing with Rust and minifilter for Windows. Its almost there. Microsoft has been working on rust driver kit.

dagmx• 3 years ago

Not surprising. Linux is one of the few monolithic kernels.

It’s kind of a worst case scenario for adding in a new language to as a result, since it can’t be added in a very isolated way.

kijiki• 3 years ago

Not surprising. Linux is one of the few monolithic kernels.

This is an amusing claim, given that Microsoft's first use of Rust in the NT kernel is in win32k.sys, which is the in-kernel code that used to live in userspace back when NT was actually a microkernel. So pre-NT4, which was released in 1996.

gjsman-1000• 3 years ago

Windows under the hood with the NT Kernel is actually regarded among some experts as technically superior to Linux with the driver model and personalities subsystem; it’s just that the stuff above the kernel isn’t so great…

LeFantome• 3 years ago

VMS under the hood

waboremo• 3 years ago

The one place microsoft's advertising branch can't get to!

yukIttEft• 3 years ago

Citations needed.

toyg4• 3 years ago

The stuff above is actually pretty good, the problems come mostly from the obsession with supporting a flawed 30-year-old paradigm (the DOS world with its simpleton prompt), a failed system-management concept (the Registry), and the recent steer towards being adware.

TazeTSchnitzel• 3 years ago

It's easier for Windows to do it because it only targets a small number of architectures (maybe only x86-64 and AArch64 these days?), and only Microsoft need to be able to build it. The Linux kernel has a huge number of targets and the build system needs to be solid for a huge number of people.

jeroenhd• 3 years ago

Windows does a lot of things in the kernel that Linux lets userspace handle. Part of that is the way UI is dealt with: for Windows UI is part of the core system, whereas on Linux it's a userspace tool you can install if you really fancy it.

There's also the advantage of being able to add Rust to the build pipeline without someone complaining that you broke the build for their TI-83+ calculator because there's no Rust compiler for that platform. Windows deals with amd64, x86, aarch64, and maybe some simple 32 bit ARM, but that's about it. Nobody is going to care about support for MIPS or Power9 or Motorola 6800 or SPARC or Xtensa.

esaym• 3 years ago

Why Rust? Doesn't Microsoft have their own similar language, C#? And then isn't there some lore about the reason Windows Vista took so long to release was due to them trying to write the entire OS in C# or something and then having to redo everything?

chungy• 3 years ago

C# isn't even remotely similar. A managed language running in a heavy VM runtime versus a direct-to-machine-code compiled language...

29athrowaway• 3 years ago

Have you heard of native AOT compilation?

lostmsu• 3 years ago

I don't think runtime is that big. The classlib is.

kortilla• 3 years ago

C# is akin to Java, not Rust.

pjmlp14• 3 years ago

Good then, given that PTC and Aicas have been seling bare metal JVMs for the last 20 years, with soft real time deployments.

nitinreddy88• 3 years ago

You should read about: https://en.wikipedia.org/wiki/Midori_(operating_system)

Tested and Failed. There are lot of interesting learnings and blogposts from this project and lot of them are actually landed in different products of Microsoft

pjmlp14• 3 years ago

Tested, and successfully run the Asian Bing network nodes during its lifetime.

If anything, it failed at the Windows business unit politics.

Unfortunely it lacked the kind of management that has pushed Java no matter what on Android, or either Swift or the high road on iOS.

CSMastermind• 3 years ago

And then isn't there some lore about the reason Windows Vista took so long to release was due to them trying to write the entire OS in C# or something and then having to redo everything?

It seems like you're confusing a few different things there.

The 'redo' step that you're talking about happened because in the early 2000s Microsoft was getting blasted by security breaches left and right.

Pretty much every senior developer in the Windows org was pulled to work on security patches. An in between update to the consumer version of Windows (codename Longhorn) was planned to hold customers over until the team had the time to do a proper iteration on the next OS.

As the senior devs were wrapping up their security fixes and coming back to see about shipping Longhorn they decided that it wasn't up to their standards and more importantly they didn't think it was moving in the right direction because it didn't include all the security fixes they had been doing.

Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over.

So they just that: they threw out everything that had been worked on, 'reset' the project to be based on the latest code with all the security fixes present, and then started the development of Vista fresh.

GeekyBear3• 3 years ago

Longhorn had essentially 'forked' XP pre-security fix and it was decided that it would be easier to throw out all of the Longhorn code and start fresh then to try and port all the security fixes over.

Microsoft internal emails from Jim Allchin (who ran Windows at the time) to Bill Gates reveal that the issue was performance.

LH is a pig and I don’t see any solution to this problem.

https://web.archive.org/web/20210427171552/http://blog.seatt...

Eventually, the solution was to drop the use of a garbage collected runtime (desired for memory safety) and go back to C/C++ (necessary for performance).

Now they are moving to a language that gives you the memory safety without taking the performance hit.

StressedDev5• 3 years ago

A small correction on the redo step.

What you are talking about was called the Windows Security Push and this is how it worked. Basically, every developer (not just the senior ones) had to help review every line of code in Windows. First, we were shown presentations which explained why security was important, where Microsoft products had failings, what were common security bugs and how to look for them. I think the presentations were done my Michael Howard. They were very good.

Then we were given a copy of Writing Secure Code to read. It enumerated all of the know types of security vulnerabilities and told us how to fix them. It also taught us how to write a threat model, validate input from untrusted sources, reduce our attack surface, use the principal of least privilege, etc.

Finally, we spent three months reviewing Vista's code. Each team was responsible for reviewing its own code. We filed bugs as we found them and then they were fixed.

Note that porting the security fixes to Longhorn from Windows XP took very little work. Windows had one code base and you could move changes from release A to release B (Windows XP and Windows Vista in this case).

Also, Longhorn was reset at some point but all of the work was not thrown out. Basically, some teams reduce the scope of their work (i.e. cut features) and some projects were cancelled. The reset did not occur because the security fixes were missing. It ocucred because Longhorn was an out of control project which had been going on for 2 to 3 years and was not close to shipping.

xedrac• 3 years ago

Maybe because it solves about 80% of the PITA security bugs that have beset Windows and other systems software since the dawn of software development? C# is garbage collected with a big runtime. It's not even an option.

pjmlp14• 3 years ago

Xerox PARC would think otherwise, pity it lost to a PDP-11 OS.

otabdeveloper4• 3 years ago

Any large company has bored people who will sneak in random languages and frameworks. The Windows network stack used to have an embedded Prolog interpreter, just because. I'm willing to bet there's a small Brainfuck codebase somewhere in the depths of Google.

StressedDev5• 3 years ago

Microsoft never tried to rewrite Windows in C# during the Vista project. I know this because I worked on Vista and my team wrote code in C++.

Vista did include some new C# APIs and Frameworks. The best ones were Windows Presentation Foundation and PowerShell (it uses a lot of .NET technologies).

Vista took a long time for a lot of reasons. I do not know all of them but my observation was there were a few basic problems:

1. Teams were allowed to check in buggy code. This led Vista to be unstable while it was being developed. Note the Kernel code was usually very good. The shell was frequently difficult to use because of the bugs and because of poor performance.

Note that when it shipped, it worked much better than it did during development. I remember being shocked at how well it worked. I also remember going back to Windows XP and realizing I actually liked Vista better (it had a better user interface, and I liked the improved Windows Update app).

2. Very poor project management. Basically, no one knew what needed to be done or how long it would take.

3. A lot of overly ambitious projects and features. Some teams really tried to do revolutionary things. Sometimes they succeeded (The Windows Display Driver Model is an example of this). Often, they failed (Media Foundation is a great example of a mediocre API which came out of Vista).

4. Some teams took dependencies on immature APIs or frameworks. The problem was the framework and the applications using it were being developed at the same time. This led to a lot of reworked because applications kept on having to be updated because their dependencies changed or were cancelled.

5. Poor leadership - Will Poole led the Window Client team (AKA the desktop version of Windows). Wille Poole previously led the Windows Media player team. My impression was he valued political skill, politics and empire building over competence and technical excellence.

After Windows Vista shipped, he was "promoted" to working on Windows for Emerging Markets. After that, he "managed" his administrative assistant. He then "retired".

I am sure I missed a lot of things and I certainly do not know everything because I worked on a small part of a huge project. Windows had thousands of software engineers, program managers, testers and managers and they worked on a lot of different things.

fifteen1506• 3 years ago

I preferred Vista interface to Windows 7.

Vista SP1 fixed most problems -- but you needed 2GB RAM.

cbozeman• 3 years ago

It's funny how you can tell who's garbage and who was kickass at Microsoft just based on what they're doing nowadays.

Dave Plummer from Dave's Garage channel on YouTube is a great example of this. Sure, just creating Task Manager alone would get you on the books for being a kickass programmer, but everything else he did, and the fun YouTube stuff he's doing now, you can tell he was really a force behind the scenes in the early days.

jen202• 3 years ago

Technically WPF was part of .NET Framework 3.0 and ran “fine” on XP, no?

(Fine meaning as well as it ran anywhere else, which was not well at all)

neonsunset• 3 years ago

C# is much more capable and fitting this purpose today than 10 years ago (perf, codegen quality and infrastructure have improved by orders of magnitude). However, it cannot compete with Rust when it comes to systems programming.

Mainly because it has different tradeoffs and its high level abstractions and other features like LINQ, interfaces, non-struct generics and async/await are very much not zero-cost, unlike in Rust.

In addition, it does not offer crucial features one would want in kernel development: deterministic memory usage, compile-time safety guarantees for writing concurrent data structures and general systems-programming-first language design.

While C# is a perfectly viable choice for writing userspace OS components, applications, UI and etc., it is a bad choice for kernelspace when C++ or Rust exist.

fulafel• 3 years ago

Rust is gaining in mindshare and C++ background curmudgeons still believe OS code can't be in GC supporting languages. But maybe it's better not to look a gift horse in the mouth, replacing unsafe code with something better still works toward slowly establishing mem safety as table stakes in OS code.

jsheard• 3 years ago

On a related note, Azure just endorsed Rust to replace C/C++ as the non-garbage-collected language of choice.

https://azure.microsoft.com/en-us/blog/microsoft-azure-secur...

Rust as the path forward over C/C++
Decades of vulnerabilities have proven how difficult it is to prevent memory-corrupting bugs when using C/C++. While garbage-collected languages like C# or Java have proven more resilient to these issues, there are scenarios where they cannot be used. For such cases, we’re betting on Rust as the alternative to C/C++. Rust is a modern language designed to compete with the performance C/C++, but with memory safety and thread safety guarantees built into the language. While we are not able to rewrite everything in Rust overnight, we’ve already adopted Rust in some of the most critical components of Azure’s infrastructure. We expect our adoption of Rust to expand substantially over time.

Gigachad• 3 years ago

Rustbros just keep winning

insanitybit7• 3 years ago

This is great but I do wish that people could see Rust as more than a better C/C++. It's an amazing language for web services and CLIs.

It's not just that it doesn't have a GC. It also doesn't have some of the worst features of OOP, it also has a sane std library, it also has an expressive type system, etc. Rust helps you write better code, even when compared to languages like Java or Python.

CGamesPlay2• 3 years ago

I agree with this but the ownership model is really a huge learning curve for somebody who wants to make an application where it's OK if it crashes when there's a bug.

It would be cool if there was a "relaxed rust" which provided interop with standard rust, but used some designated smart pointer for everything automatically (at the crate level, for example). I'm not sure how/if this could work, but I think it would make transitioning from GC languages -> Rust much easier.

qaq5• 3 years ago

Any sign of async functions in traits without the macro? One kindah would expect first class async support for web services and honestly something like Swift prob. has better balance between usability and speed for most web services.

toyg4• 3 years ago

The trade-off in complexity and loss of productivity (particularly at prototype time) are just not worth it.

kaba09• 3 years ago

It’s as amazing for web services as C++. There might be some niche use case, but would not be my first, nor second choice there.

It is still a low-level language, and while it is expressive, that doesn’t stop the low-level constructs from leaking to high level abstractions — which is not a negative in itself, that’s the purpose of the language. It’s just a hard tradeoff when it comes to anything not needing low level considerations.

pharmakom5• 3 years ago

All true but it’s not as productive as a GC language from the ML family. If you don’t need the performance / latency guarantees I’m not sure it’s worth it.

pkolaczk2• 3 years ago

Rust can have some bad effects on your brains. I noticed I can't read Java code as easily as before. E.g. yesterday I had to read it multiple times before I understood what was going on and why they are calling setters on an item that goes out of scope:

    List<Something> list = ...;
    for (...) {
        Something item = new Something();
        list.add(item);
        item.setFoo(...);
        item.setBar(...);
   }

DrBazza2• 3 years ago

> Rust as the path forward over C/C++

I hope not, but it's the popular memory safe language we have now, and ironically Stroustrup's quote applies "There are only two kinds of languages: the ones people complain about and the ones nobody uses".

I would like Val to succeed: https://www.val-lang.dev/ With a Swift/Kotlin style syntax, the learning curve is likely easier as well.

usr11062• 3 years ago

Shows up? I haven't used Windows for close to 15 years, but I have not heard that their kernel would be open source now.

dang2• 3 years ago

Ok, we've made the title not show up above.

usr11062• 3 years ago

Interesting. I thought editorializing titles is not accepted on HN.

My criticism was directed towards the article, not the submission.

Edit: Ah, the guidelines say unless it is misleading.

Well, I would not have called it truly misleading, just typical headline style you can often see. I did not like it, but I did not feel misled.

sylware• 3 years ago

Huh? linux is the windows kernel or what?

badrabbit• 3 years ago

Win32k.sys in rust would kill so many future bugs. RIP.

criddell3• 3 years ago

How’s Rust for desktop application development? Are there frameworks like MFC or ATL or WTL or WinRT? Is COM development or using UWP significantly more difficult in Rust than C++ or C#?

jenadine• 3 years ago

There are ok bindings to some toolkits such as Qt and Gtk, which are the most mature right now. But there is promising progress also in full rust GUI libraries for example Slint or egui

criddell3• 3 years ago

I should have mentioned I’m only interested in native Windows APIs. I want to use UWP features, lots of COM (both as client and server), etc…

I see Microsoft has done some work and there’s a Windows crate, but I’m not seeing IDL support for COM and it looks like you end up with a ton of unsafe blocks. It makes me wonder if a programmer who sticks with smart pointers in C++ is all that much worse off than a Rust developer on Windows?

hawk_• 3 years ago

Good question. Specifically GUI code tends to have many cycles in object graphs. Do idioms already exist in Rust to deal with advanced cases of that?

First Rust code in the Windows 11 kernel

Related Stories