One Windows Kernel

https://techcommunity.microsoft.com/t5/Windows-Kernel-Internals/One-Windows-Kernel/ba-p/267142

by MikusR • 8 years ago

305 215 8 years ago

techcommunity.microsoft.com

detuur2• 8 years ago

The most amazing aspect of all this is that the core of Windows, its kernel, remains virtually unchanged on all these architectures and SKUs.

Is that really such an exciting concept? Maybe I have a poor understanding, but I've been under the impression that nearly every competing kernel (Linux, Darwin, *BSD) is exactly like that. The only situation where I can imagine a slight departure is Solaris and its SPARC architecture, where SPARC had certain features to specifically to tie in with the kernel but I'm sure it still compiled to foreign architectures from the same codebase.

ChuckMcM• 8 years ago

Well it is an exciting concept but it isn't as you point out unique.

It was fairly clear in the 80's and 90's there were two 'streams' of systems programmers, those who started at mainframes and mini-computers and were making things work on smaller and smaller versions of those machines, and those who started at the ROM monitor or DOS level type machine and made things work on larger and larger versions of those machines.

It was stark how differently the two 'types' of people approached operating system problems.

Windows "crossed over" from being more of a monitor type interface to being an executive interface with Windows NT at the hands of a former VAX software architect (which was firmly in the mainframe getting smaller path), and that cross over happened for the masses when Microsoft merged the NT kernel with the Window "old tech" kernel in Windows XP as I recall. By the time Windows 7 came around it was pretty much unified for life and has evolved since then.

If all you knew were Microsoft OSes, then yes it is an amazing journey. If however you had already been experiencing systems like UNIX that ran on everything from mainframes to PDP-11/70's you would not have been quite so surprised.

smacktoward• 8 years ago

That's the core irony of the PC revolution. PCs started out with the explicit goal of being something different than mainframes and minis. But when people started networking them together, the only way to keep them viable was to essentially turn them into mainframes and minis, just in different shaped boxes. So we've gotten to live through 20 years of PC vendors reinventing things that big-iron users had access to long ago.

cptskippy3• 8 years ago

The evolution of DOS and Windows over the last 30 years makes me wonder if Windows is really even Windows.

It started out as a Windowing environment that ran on top of DOS but with Windows 95 it switched to a Windowing OS that also ran DOS. Later all of that was replaced with an entirely different OS (NT/XP) that basically supports the legacy OS in emulation.

e12e2• 8 years ago

Maybe something happened with xp too, but "windows 2000" was the departure to "NT kernel" land for desktops/home users AFAIK. (I guess xp was really 2000 with windows 95 heritage bling added).

Maybe they did try to force windows millenium on poor saps parallel to w2k - and didn't quit that racket until xp?

blattimwind• 8 years ago

If however you had already been experiencing systems like UNIX that ran on everything from mainframes to PDP-11/70's you would not have been quite so surprised.

Back then (R) there wasn't one UNIX, rather everyone had their flavour of it for their kind of machine. I don't think source compatibility became mainstream until the late 80s to 90s, before that it was all ports.

markbnj• 8 years ago

> at the hands of a former VAX software architect

Dave Cutler. I think he is still a Senior Technical Fellow at MS and a lead developer on Azure.

dasmoth• 8 years ago

At the time this switchover was happening, I remember being pretty excited about things like protected memory. But in retrospect, I think it’s a shame there hasn’t really been a continuation of the “monitor” type interface. I could imagine them winning at things like minimising latency when responding to user input.

tinus_hn• 8 years ago

The Windows NT kernel always was Cutlers neat and tidy core with ugly parts bolted on because they somewhat made sense at the time. These parts are integrated pretty deeply so it’s quite hard to use this kernel in a system that’s not Windows with a desktop.

So although it’s a race for Microsoft against Microsoft but it’s still somewhat of an achievement they ended up with this result.

partiallypro• 8 years ago

It's an exciting concept within Microsoft and the Windows team and could bring good things. But it's not a unique concept, no.

rayiner2• 8 years ago

Solaris doesn't scale down to cell phones, and Darwin doesn't scale up to 896-core servers. Linux does scale like that, but Linux is pretty unusual in that regard too.

detuur2• 8 years ago

That is actually a really good point. At the end of the day there are many, many kernels out there and only Linux, BSD, and NT scale from all the way at the bottom to all the way at the top. I actually even recall my scepticism when MS announced Windows for IoT platforms (not that this scepticism has abated, though).

pjc50• 8 years ago

Solaris hasn't been scaled down to cellphones, but remember that today's cellphone is yesterday's high performance workstation. A Raspberry Pi outperforms a SPARCStation20: http://eschatologist.net/blog/?p=266

Windows also used to scale down quite well; I'd love to know how much really was common between NT and CE.

CyberDildonics2• 8 years ago

Are there servers with that many cores? I only know about Xeon Phis and 8 socket xeon servers that go up to around 160 cores / 360 threads/logical cores

icedchai• 8 years ago

I'm skeptical. In the 90's, Solaris ran on boxes with 16 megs of ram, just like NextStep. It could easily scale to run on today's phones... if there was a reason.

gnu8• 8 years ago

There are still a lot of people out there who don’t understand that there are computing environments outside of Microsoft Windows. Not as many as there were in the 1990’s, but still quite a few.

lizknope• 8 years ago

Sun had different kernels for the sun4c and sun4m architectures.

In this infamous post David Miller brags about how he has made a single Linux kernel capable of booting on both.

https://cryptnet.net/mirrors/texts/kissedagirl.html

and then the response from Bryan Cantrill

amanaplanacanal2• 8 years ago

That is freaking hilarious.

mschuster913• 8 years ago

Is it possible to "backport" the Kernel (and other parts, such as the WSL, with it) to Windows 7?

I'd like to have a modern Windows, but with the "classic" UI and no forced ads/bloatware/crapware. And yeah I am aware of W10 LTSB but it's not legally available for 99% of users.

edit: Looks like the post attracted a bit of a flamewar. Originally, I just wanted to know if this was technically feasible or not.

rpeden• 8 years ago

If you had the source and enough developer time...sure, it would be possible. :)

212• 8 years ago

Same old story.

When XP came out, people were moaning that it looks like shit and how they wanted the XP kernel, but the 2000 look.

When Vista came out, same thing again, all that Aero looks like shit, disable it so that it looks like XP classic.

Now 10 comes out, same story, looks like shit, we want it to look like 7.

You should think deeper to see what drives this desire of yours. I bet that when Vista came out with the Aero look, you felt like it's shit forced on you, and you wanted the XP classic look. Now you want the Aero look that you once hated. Ironic, isn't it?

cptskippy3• 8 years ago

It's interesting though that no one pined for Vista or 8.

jdhzzz• 8 years ago

I could live happily in on a system that looked like Win2k.

mschuster913• 8 years ago

    You should think deeper to see what drives this desire of yours.

Decades of using it (21 years, by now), plus the total dislike to any kinds of advertising IN A PRODUCT I PAID FOR. Additionally, people had the CHOICE to keep Windows looking, basically, like Win95 - until Windows 8/10 came along. With Win10 that's not possible any more. It treats me like I'm either a child or using a touch device, and hell no. I want a clear optical distinction between a mobile and a desktop/laptop OS, alone because the operating mechanisms (mouse vs touchscreen) are so different.

Who in their right mind thought people would touch on their big-time office PC screens and forced the UI to this absolutely crappy (for desktop!) tile stuff?

Oh and don't get me started on the forced telemetry crap and the forced / silent updates and the reverts that these updates "accidentally" do once you managed to render your PC free of this crap. Microsoft has completely lost my trust by now, I switched happily to an Apple environment. At least it doesn't force Candy Crush upon me or injects ads in the Start Menu and Apple vendors (well, to be fair, there is only Apple itself) don't preload my machine with crap I don't need (a dozen updaters punching security holes in the system, "free" virus scanners, spyware)!

titanix2• 8 years ago

Windows 10 UI is objectively bad on a lot of points like consistency, search from the Start menu, etc. At least up until Win8 you could switch to the classic theme to get something looking like 98’s UI. Now you have no choice but to suffer the big button dialogs made for tablet on any laptop / desktop without touchscreen.

ConceptJunkie2• 8 years ago

But Windows XP, Vista and 7 _could_ look just like Windows 2000. Starting with Windows 8, that is no longer an option.

I can't speak for anyone else, but I thought every version of Windows since 2000 looked like garbage by default. The difference was that you used to be able to do something about it.

kitsunesoba2• 8 years ago

Win2K to this day remains my favorite Windows release. It was just so clean and practical. It had its frustrations (setting up a Win2K/WinXP/Win7 system from scratch was always a task and a half, with all the driver installs and updates involved) but once you got it set up it was great.

XP/7 aren’t bad but they definitely felt more gimmicky.

nine_k3• 8 years ago

It appears that MS made a major design effort when they designed the "Windows 95" look and feel. They produced a number of coherent concepts and tried to stick to them everywhere. They kept the amount of fluff at bay.

It also appears that since then, they've never made an equivalent re-design effort to produce a coherent design and stick to it. XP was mostly an incremental change + re-skinning. Win 10 looks haphazard, unfortunately, and, most of all, the new UI is only applied sparsely, so you can't expect it to be equally usable everywhere.

A Linux desktop circa 2000 could have a hodge-podge of GUI programs based on of GTK, Motif, and Athena widgets. But I could not expect the same in 2018 from Microsoft's flagship desktop OS.

InclinedPlane2• 8 years ago

When 7 came out people said "hallelujah you listened!" even though it wasn't perfect. When 8 came out people said "why did you do this to us?" When 10 came out people said "at least it's better than 8." Windows 7 was probably the high point of actually doing the work to make the OS usable and clean instead of bloated and annoying. Before that Windows 95, for all its flaws, was built on a ton of actual real world usability research. Unfortunately, MS has a long history of ignoring fundamental research and common sense and dictating "the way things will be" with the OS UI/UX. They think this is what people want since Apple does it and gets away with, but MS isn't Apple.

Also, the fact that people keep complaining doesn't mean that those people are wrong, it's possible for a company to continue fucking up year after year for decades. I could explain exactly why MS makes these mistakes (part of which is due fundamentally to bad corporate culture and a long legacy of bad incentives and performance review systems) but that's another conversation.

vbezhenar2• 8 years ago

Of course it's possible. Windows UI is just set of userland programs. Back in Windows XP time, there were plenty of Windows shells which completely changed the way OS looks. I'm not sure if those programs are still in development, but I don't see any particular reason why it would be impossible to replicate Windows 7 L&F for determined developer.

wernercd2• 8 years ago

http://www.classicshell.net/

First thing I install on windows 10 (via ninite.com) and followed by removing everything I can (and some things I can't like Cortona)

Others shell changers exist, but this is the one I use.

cududa• 8 years ago

I had a startup attempting to build a Windows shell. It’s no longer possible. You could do it with 7 with some very clever work. Geohot came on around the time Win8 was in beta and figured out some extremely clever ways to make it work on that. But by 8.1 a few key elements to make a decent shell became signed functions and you’d have to perpetually hack the UWA sandbox which is not feasible.

basch4• 8 years ago

https://www.deviantart.com/kizo2703/art/Windows-classic-them...

alkonaut• 8 years ago

I’m guessing it’s easier to strip out the things you don’t want from Win10 than to transplant a kernel back to Win7.

yuhong• 8 years ago

Barnacules just told me that he worked on the Windows 8.1 start menu update just before he left in ~2014 before it was cancelled in favor of Windows 10. Anyone remember this, BTW: https://news.ycombinator.com/item?id=7774476

gfody• 8 years ago

it's possible to install Server Core (w/the "desktop experience" component permanently stripped out) and then run something like https://github.com/cairoshell/cairoshell - there's a feature request to get ClassicShell running on Core (https://github.com/Open-Shell/Open-Shell-Menu/issues/44) which would be a great way to get the classic look-and-feel on the modern kernel

userbinator• 8 years ago

I've not looked into it in detail (I used to be a "hacker" who spent lots of time disassembling, patching, and otherwise modding Windows in the 98/2K/XP era) but I read that the telemetry stuff is deep in the kernel too, so you probably don't want a "stock" Win10 kernel. Vista and above have various DRM-ish obstacles that get in the way, and the near-continuous updating of the newer versions also frustrates any efforts. Probably against the EULA too.

That said, there exists leaked source of an XP/2k3 kernel and people have managed to compile and use it with an XP userland. Anything is possible if you have the time, skills, and the complete lack of concern for EULAs. ;-)

MSFT themselves open-sourcing large parts of Windows in the future, including the kernel, would not be surprising either.

Another, possibly less legally questionable approach, would be to try getting the ReactOS/WINE userland to run on the Win10 kernel.

tialaramex• 8 years ago

This seems like it's essentially confessing to something we already knew. Back at the turn of the century Microsoft liked to claim that Windows Server and the ordinary client NT were different in some important way beyond just the branding and price. People would even try to argue that Linux couldn't be a "real" desktop OS because it lacked some kind of secret sauce not found in a server OS... Hackers eventually found out how to tell a client NT kernel to behave exactly like Server, because of course they are the same kernel, duh.

Microsoft's reaction was to stop selling the two next to each other. There is no Windows 10 Server and Windows Server 2019 doesn't have a client version. So the exact same thing continues, but now there IS no equivalent product to prove the point that they're identical bar branding and price.

basch4• 8 years ago

Microsoft kinda gave up on this ruse back at Longhorn and Vista SP1 days.

Longhorn was a fork of the Server 2003 codebase, and Vista SP1 was a backport/newfork of changes made to Server 2008. (Server2008 and VistaSP1 were both released Feb.4.08.) They were absolutely released in lockstep. Vista SP2 was released April.28.09 and Server 2008 R2 was released Oct.22.09. At that point the consumer product became the testing ground for the server kernel.

It's easier to think of Vista RTM as a giant public alpha, and and Vista SP2 as an insider preview for Server2008 R2 / Windows 7.

dblohm72• 8 years ago

Hackers eventually found out how to tell a client NT kernel to behave exactly like Server

It was Mark Russinovich who did it with a utility called NTTune. Now he's the CTO of Azure!

abainbridge• 8 years ago

You mean Mark Russunovich the sci-fi author: https://www.amazon.co.uk/Rogue-Code-Jeff-Aiken-Russinovich/d...

That Azure stuff is just his hobby.

dblohm72• 8 years ago

Here is the archived piece from their old ntinternals.com site:

https://web.archive.org/web/19970605083609/http://www.ntinte...

kyberias• 8 years ago

That was always a bogus and boring argument. Was then and is now.

I don't think Microsoft ever claimed that they have a different kernel. It was always clear they have.

Windows Server includes a lot of OTHER SW, other than Kernel, that makes it a server when compared to desktop Windows.

Server and Desktop Windows are like different Linux distros.

dboreham2• 8 years ago

I've worked (outside MS) for 20+ years with NT, beginning with the 3.1 beta. It is news to me that anyone believed the kernels were different. I can't imagine why anybody in the least bit clueful would believe that. Furthermore I've never heard anyone suggest or argue that they were different.

mixmastamyk4• 8 years ago

Believe the main point of contention was a 10 client limit to the services of NT Workstation. MS argued that Workstation was not able to handle more due to fundamental differences, but those with a clue suspected it was merely a registry setting or two. Not to mention it could handle that many web clients (at least) without breaking a sweat, but licensing prohibited it in theory.

Related, there's been a switch in the System control panel to prioritize services/interactive use, though it has been papered over by two layers of cruft in recent versions so is hard to find.

bhauer• 8 years ago

It is news to me that anyone believed the kernels were different.

Indeed. At boot time, the version of the kernel was displayed on the blue screen above the progress dots, and it was the same on Workstation and Server. If I recall correctly, the important distinction was between the uni-processor and SMP kernels.

zeptomu• 8 years ago

So ... how long until Windows becomes just a Linux distribution similar to Android? 8 years? Sooner, later?

pjmlp9• 8 years ago

Windows is most likely to become another mainframe like OS, with its multiple execution personalties than yet another a Linux distribution.

Just like IBM i, IBM z, ClearPath are their own thing, and can also run Aix, Windows and GNU/Linux on top.

Sharlin2• 8 years ago

I wonder what it would cost to purchase an Oracle license for that 1792 core instance.

linuxlizard• 8 years ago

We'd probably need 4 cores just dedicated to that math.

teddyh• 8 years ago

The answer is: Whatever amount of money which Oracle’s sales and customer research teams determines that you can barely afford to pay.

ianmcgowan• 8 years ago

Aka "how deep are your pockets?". The flip-side to this comes at quarter-end, when the question is turned around on your sales rep - "how far are you from quota?". Some amazing deals when they just need a few deals to make their number...

TimTheTinker5• 8 years ago

It seems to me like Oracle has figured out a sort of secret formula for making a lot of money through shrewd business practices while managing to avoid the public hatred Microsoft and IBM attracted when they did likewise.

Perhaps because they market to business leaders who could stand to lose a lot if Oracle unleashed the legal hounds?

oraclethrow• 8 years ago

Oracle reseller here:

~59.2M USD on first year (includes maintenance), assuming: - Xeon processors core factor - OEE without any options (no RAC, no partition, etc..) - List price

22% yearly maintenance, about 10.8M on second year and forward.

You won’t pay list price, most likely.

TimTheTinker5• 8 years ago

Consider how much less you’ll spend hiring a whole team of the best DBAs to set up and maintain a competing OSS system.

SEJeff2• 8 years ago

    Windows is one of the most versatile and flexible operating systems out there, running on a variety of machine architectures and available in multiple SKUs.

Linux, BSD, Darwin... You support a dozen architectures? How adorable. condescendingly pats head

cyphar3• 8 years ago

You support a dozen architectures?

And they only support four (x86, amd64, ARM, ARM64)! This is quite a cute thing to mention after saying you're "one of the most versatile and flexible operating systems". Ignoring the obviously qualifying language of "one of the most", I would argue it is still false -- virtually every mainstream operating system I can think of supports more architectures than this. Here is my quickly-put-together list:

    * Linux: 25.
    * MINIX: 15.
    * OpenBSD: 13.
    * FreeBSD: 7.
    * Windows: 4.
    * illumos: 3 (with arm* in development).
    * Darwin: 2.

What an achievement. Now -- don't get me wrong, Windows has done a lot of things, but they have set the parameters of this comparison and they don't fare well in it (even if you get past the qualifying language over a comparison that they decided to make).

SEJeff2• 8 years ago

NetBSD: 57!

cestith• 8 years ago

Certain whole Linux distros would be high upon that list, not to mention NetBSD.

* NetBSD: 8 in Tier I, another 49 in Tier 2, total of 57.

* Gentoo Linux: 10 to 12, as they target i[456]86 separately within IA-32

* Debian Linux: 11

* Fedora Linux: 4 primary, 7 secondary, total of 11

* L4: 9

* Red Hat EL: 8

* Ubuntu Linux: 8

* QNX OS: 8

* OpenSuSe Linux: 7

* Alpine Linux: 6

* CentOS (Linux): 5

* Illumos: 4

* Kali Linux: 4

* Q4OS (Linux): 4

* Android: 4

* Arch Linux: 1 official, 2 unofficial

* Parabola Linux: 3

* Slackware Linux: 3

* ChromeOS: 3

* QNX Neutrino RTOS: 2

* Haiku: 2, with 3 more ports underway

RiscOS runs across four or five generations of ARM.

Darwin used to run on three more platforms but newer versions don't.

Illumos is the kernel and libraries underlying OpenIndiana, Dyson, and many other systems based off the old OpenSolaris effort.

kccqzy• 8 years ago

Darwin supports 4, I thought. In fact the same four as Windows.

gaius3• 8 years ago

they only support four (x86, amd64, ARM, ARM64)

To be fair, NT shipped on x86, PPC, Alpha and MIPS. There was a never-released version for SPARC. It was originally developed on the i860. Wasn’t later Windows also available on Itanium too?

MS cut down on their list of supported CPUs for commercial, not technical reasons.

karmenblack• 8 years ago

I wish Windows kernel been optimised for a given architecture. It is one of the examples where "one fits all" doesn't work. Even if Intel or AMD bring another grounbreaking generation of CPUs - giving mighty 5% increase in performance - it is all for nothing, because it gets eaten up by Windows poorly written guts. I don't know what kind of monster machine you need nowadays to watch YouTube video without audio breaking up or without having to dedicate one core just to handle the mouse.

matachuan• 8 years ago

C'mon. Not even your own Drawbridge project is supported in Windows...

pjmlp9• 8 years ago

Have a look at WSL pico-processes, device guard, and secure kernel support.

andrewstuart• 8 years ago

I wish they had the courage to get rid of the registry.

stephengillie• 8 years ago

What would replace it? Thousands of individual config files? That's a mess to manage.

What the Windows Registry needs is a REST API.

Jedd• 8 years ago

What would replace it? Thousands of individual config files? That's a mess to manage.

That's not necessarily true.

In the 90's I worked at a large telco where we'd gotten good user roaming functionality along with package management using third party tools, for about 1600 applications, on Windows 3.

Uninstalling a package demonstrably left no remnants behind.

The migration to Windows 95 was troublesome partly because it was much harder for roaming users, but mostly the registry was much more difficult to manage - Office applications were notably impolite, with thousands of registry entries, many of which lingered after an uninstall, and no way to sanitise the registry other than reinstalling the OS.

If you're using a computer without a package management system I could see how you could be convinced config files are harder to manage (by hand) than an opaque blob that poorly hides its ugliness, but you'd have bigger problems.

gruez• 8 years ago

What the Windows Registry needs is a REST API.

shudders

badpun• 8 years ago

What the Windows Registry needs is a REST API.

Why would you want a networked API for stuff that should be internal to the machine?

orf15• 8 years ago

And replace it with?

CyberDildonics2• 8 years ago

Replace it with an SQLite db for OS settings and make programs save their settings in files so they are contained within a single directory.

MisterTea2• 8 years ago

How about a file system? Because that is precisely what the registry is, a file system in a file. The keys are just paths to a variable which can be a text or binary file.

And people who suggest SQL or another monstrous abstractions as an implementation need to take a step back and apply the idea of philosophical simplicity to their work. I honestly don't understand why people have this tendency to over engineer simple problems. I suppose it helps during review time.

viraptor2• 8 years ago

Why? What's the goal / advantage?

cm2187• 8 years ago

Actually they are the first to complain about the registry. The problem is that an app will make some changes all over the registry so it hard to uninstall it cleanly or control the state of the machine. They are considering exposing a virtual registry to an app and saving the diff the app applies and stuff like that.

rustcharm• 8 years ago

Most applications today store their preferences in an AppData directory that can be per user, per machine, or per network. The registry is used largely for OS settings.

urda• 8 years ago

The Windows Kernel is an insane engineering feet by itself, so getting to see how the inner components mesh in articles like these always fascinates me.

partycoder• 8 years ago

Forgot to add the line count of backdoors and stupid absurd shit like processing fonts on the kernel rather than userland so people can hack your 896 core machine and mine bitcoin.

crumbshot• 8 years ago

Your claim of backdoors is nonsense of course, but the kernel mode font handling was indeed a security risk. However on recent releases of Windows, they changed font parsing to run in a user mode sandbox: see the "Mitigating font exploits with AppContainer" section of https://cloudblogs.microsoft.com/microsoftsecure/2017/01/13/...

antiwin32• 8 years ago

How is it possible to have CPU with 100% of usage and Memory at 0% of usage at this screenshot (Task Manager showing 1792 logical processors)?

SteveCoast• 8 years ago

It was a 2Tb of RAM machine I think? So 0.49% of that would be 9.8Gb? Plenty of space and it would still be "0%" used.

irfanhamid2• 8 years ago

How come out of 1TB 9GB is 0%?

saagarjha3• 8 years ago

I don't see why not: create a process that just spawns a thousand threads that infinitely loop, but uses very little memory.

irfanhamid2• 8 years ago

That still is some memory. Threads don't run without consuming memory and some are prone to memory leaks. But, that's not the point here.

Annatar• 8 years ago

"This is a picture of Windows taskmgr running on a pre-release Windows DataCenter class machine with 896 cores supporting 1792 logical processors and 2TB of RAM!"

Reminds me of an UltraSPARC T5 server with 3072 processors I used to work on... from five years ago. Old tech, eh?

One Windows Kernel

Related Stories