Windows NT/OS2 Design Workbook

https://computernewb.com/~lily/files/Documents/NTDesignWorkbook/

by markus_zhang • 5 months ago

143 62 5 months ago

markus_zhang12• 5 months ago

I think this is the only place that hosts a (hopefully) full electronic copy of the book.

jdboyd• 5 months ago

For those unclear on what the large pile of .doc and .pdf files are, it appears to be some revision of the design documents for "NT OS/2", which then turned into just NT. This appears to be the Smithsonian description of their physical copy: https://www.si.edu/object/microsoft-windows-nt-os2-design-wo...

markus_zhang12• 5 months ago

Yeah I think this is the original design doc for NT.

cpeterso• 5 months ago

The first release of NT is planned as a workstation product that will provide a strong competitor to UN*X based workstations.

UN*X spelling for trademark reasons or a joke that UNIX is verboten at Microsoft?

AshamedCaptain• 5 months ago

It is a generic way to refer to unix and unix-like systems. It is still in use today, e.g. to indicate Linux as part of the set. For this document most likely it refers to Xenix (MS's unix).

moron4hire• 5 months ago

Eunuchs is already a plural word.

cmiles74• 5 months ago

I think trademark, I remember Bill Gates referring to Windows NT as "a better UNIX than UNIX".

p_ing2• 5 months ago

OS/2 is likely where that came from - https://www.theregister.com/2024/02/21/successor_to_unix_pla...

anthk3• 5 months ago

That's Plan9 and 9front. I'd say NT it's a better VMS, VMS++ in the alphabet ;), as you get V->W, M->N, S->T, so "WNT".

fredoralive2• 5 months ago

IIRC Microsoft's internal email still ran on Xenix at the time (until Exchange betas got good enough for internal use c. 1995?), so perhaps more trademarks than some sort of absolute hatred of Unix. Also note that one of the two APIs that NT OS/2 was initially going to support was POSIX, albeit perhaps more because the US government wanted that than a true love of UNIX. Although the design rationale document (ntdesrtl) does lament that existing POSIX test suites tend to also test "...UNIX folklore that happens to be permissible under an interpretation of the POSIX spec".

nucleative• 5 months ago

Did Microsoft never run Microsoft Mail internally?

It was an email system that ran on top of file system. If I recall, mail clients connected over a networked drive to access mailboxes. So it was never regarded as being very scalable.

kryptiskt2• 5 months ago

That was back when there was "real" UNIX around, as well as a number of clones, including Microsofts own Xenix (maybe they had offloaded that to SCO by then). So UN*X was one way to indicate that it meant UNIX-like OSes.

kryptiskt2• 5 months ago

Turns out SCO bought Xenix in 1987, but Microsoft was just a couple of years removed from being the biggest Unix vendor around at this point.

vikingerik• 5 months ago

As others are saying, the * is meant as a wildcard, not as censorship. It's meant to also cover the likes of Linux or Xenix etc, although there isn't actually any other name that would strictly fit the pattern of "UN*X".

themafia2• 5 months ago

So.. you had _all_ this.. and for some reason just didn't want turn it into a useful set of "man" pages in your OS?

If they had their eye on the actual ball they wouldn't need to write Halloween memos and rant about developers on stage.

fredoralive2• 5 months ago

This mostly describes stuff to do with the [Windows] NT [OS][/2] (delete as appropriate) kernel layer, which normal mere mortals aren't supposed to interact with. You're supposed to use stuff like the Win32 KERNEL32.DLL not the more direct DLL, NTDLL.DLL (a DLL). Of course, true hackers scorn such abstractions.

p_ing2• 5 months ago

Microsoft's eye wasn't on open sourcing their OS and describing the deep internals. They still don't want you to develop against the NT API, even though developers certainly do (and Microsoft makes compatibility shims for applications which do, when required).

torginus• 5 months ago

Tbh, I think Windows' stable ABI/API at all levels + excellent docs is something the open source community could learn from.

Software that is poorly documented, has no stable api, and no way of accepting extensions other than modifying the source, is not very 'free', in the sense, that if I have a good idea, I'll have crazy amounts of trouble getting the change to users even if I manage to decipher and modify the source appropriately.

This approach describes many software projects. If I wanted to change GCC in a way the original authors didn't approve of, I'd have to fork the entire project and lobby the maintainers to include my versions.

If I wanted to change most MS stuff, like Office, I could grab the COM object out of a registry, and override it with mine, just to list one example. Then I could just put the binary on my website, and others would be able to use it.

As for MS not publishing these particular docs - its not like MS doesnt publish tomes on everything, including the kernel. If you're interested in Windows Internals, I recommend the book series of the same name by Pavel Yosifovich, or really anything by him or Mark Rusinovich.

Its also a testament to the solidity of Windows' design that tons of stuff written decades ago is still relevant today.

markus_zhang12• 5 months ago

I think Windows and DOS do have good documents. I actually think they had way better ones than Linux at the time. But I could be wrong.

For reference:

https://jacobfilipp.com/msj-index/

And also MSDN.

bombcar3• 5 months ago

Windows programming guides provided by Microsoft were simply amazing for the time; the documentation available was excellent.

Part of the reason they did so well, companies could easily implement software using the new APIs.

Of course, they also had secret and undocumented APIs that people found and wanted to use ...

themafia2• 5 months ago

Was MSDN always free? I remember their access prices for the services in the late 90s and early 00s were eyewatering.

delta_p_delta_x2• 5 months ago

I have a hot take. manpages are really bad for noob examples, or if you actually want to learn how to use something. They are great references if you already know 95% of the tool, but for the most common use cases, they completely lack any sort of examples.

In this sense, LLMs (as much as I am sceptical about them) are much more useful.

markus_zhang12• 5 months ago

Yeah I agree, TBF I rarely found man-pages to be useful to me, while so far LLM is pretty good at bash scripting, at least at the level that I need. But of course still wants to learn this stuffs in depth.

asveikau• 5 months ago

This depends a lot on what manpage you're looking at.

When I learned C more than 20 years ago, I found libc manpages a pretty good way to learn. For many functions in section 3, you can read the manpage and make an intelligent guess on how it's implemented, and write your own implementation. I did this as an exercise back in the day.

pdw• 5 months ago

In the early 90s Microsoft distributed the full Win32 API documentation as a WinHelp file. It felt very much as hyperlinked man-pages. Super fast too, even on machines of the time. WIN32.HLP can still be easily found, but modern Windows versions no longer ship WinHelp :(

phendrenad2• 5 months ago

This is giving me flashbacks to the times when I had to implement systems based on big, verbose specification documents like this. Horrible.

If you really want to understand Windows, skip this and check out Windows 2000 Internals by Solomon and Russinovich (Win2k is a good middle-ground where Windows had matured a bit).

markus_zhang12• 5 months ago

Yeah this is definitely a hell of read which probably has more historical values than real ones. The book "Inside Windows NT" also gives a good overview about Windows NT 3.1. There is another one called "Understanding Windows NT File System" which talks about the fs I think.

The biggest problem is NT is not open-source, and while there are leaked copies posted online, there is no "official" build guide so people have to try their luck.

delta_p_delta_x2• 5 months ago

For more modern Windows, Windows Internals Parts 1 and 2[1][2], and Windows 10 System Programming Parts 1 and 2. Basically anything by Yosifovich, Russinovich, Ionescu et al.

[1] https://www.microsoftpressstore.com/store/windows-internals-...

[2] https://www.microsoftpressstore.com/store/windows-internals-...

[3] https://leanpub.com/b/windows10systemprogrammingP1and2

RattlesnakeJake• 5 months ago

It's giving me flashbacks to a few months ago.

(*cries in X12 270/271*)

Lammy• 5 months ago

My favorite part of NT is the Local Procedure Call (now obsoleted by ALPC): https://computernewb.com/~lily/files/Documents/NTDesignWorkb...

Very cool to be able to read the original design instead of just reverse-engineered ones. Thanks for posting!

kh9000• 5 months ago

The section coding.pdf has their code style guidelines, colloquially known as Cutler Normal Form, CNF for short. I'm conflicted on it. Definitely overly verbose, but you can't argue with the results of the NT team. Such a rigid style guide almost feels like the technical version of a dress code. And there's an idea called "enclothed cognition" which is like, if you wear a business suit to work, it exerts a subconscious influence that results in you taking the work more seriously, focusing your attention, etc: https://en.wikipedia.org/wiki/Enclothed_cognition

markus_zhang12• 5 months ago

I agree that in general people are influenced by their perception of themselves. For example I always pretend I'm a kernel programmer when hacking on small kernels. This did lend me a lot of patience to debug and figure things out, which I do not have for my work.

bombcar3• 5 months ago

It's also important to remember that a ton of things we take for granted now simply didn't exist (source code control was in its infancy, merging was shit, syntax highlighting was minimal at best, compiling took time, etc).

anthk3• 5 months ago

Not with Borland products. Even XEmacs and Emacs had these features (code control was with CVS or close).

jen20• 5 months ago

Source control was in infancy compared to today, but still 15 years old (SCCS) when Windows NT development started!

nxobject• 5 months ago

At least there was this...

Note that the NT OS/2 system does not use the Hungarian naming convention used in some of the other Microsoft products.

Eisenstein• 5 months ago

I particularly like this bit in ntdesrtl.pdf:

"6.1 OS/2 Standards

Our initial OS/2 API set centers around the evolving 32-bit Cruiser, or OS/2 2.0 API set.

(The design of Cruiser APIs is being done in parallel with the NT OS/2 design.) In some respects, this standard is harder to deal with than the POSIX standards. OS/2 is tied to the Intel x86 architecture and these dependencies show up in a number of APIs. Given the nature of OS/2 design (the joint development agreement), we have had little success in influencing the design of the 2.0 APIs so that they are portable and reasonable to implement on non-x86 systems. In addition, the issue of binary compatibility with OS/2 arises when the system is back-ported to an 80386 platform.

This may involve 16-bit as well as 32-bit binary compatibility."

Very "professional" coded writing that expresses a frustration with the need to collaborate with IBM that could have been more succinctly written if they had the option to use a few choice four letter words.

JdeBP• 5 months ago

And also not entirely correct. The 16-bit 1.x API was definitely x86-specific. But the 32-bit 2.x API was not, as evidenced by OS/2 for PowerPC actually existing at one point.

anthk3• 5 months ago

The layout it's close to /sys/doc for 9front/plan9 with guides for Acme, Sam editors (the guide to structural expresions as a tutorial and a guide), the Rc shell, the kernel, the ACID debugger, FS' (several, fossil/venti, cwfs and now GeFS), the security, the compilers, the Rio window system, namespaces, processes and channels, plumbing (something like xdg-open but far easier and more powerful), ... much more than a closed source and propietarty NT implementation, as you get the source for free too under libre licenses. Thus, it makes debugging the system a far easier task. And it's far simpler than Unix itself too.

hulitu• 5 months ago

Windows NT had a design ? It all looks like a bunch of staff thrown at a wall in the hope that it will stick.

markus_zhang12• 5 months ago

The core NT team came from DEC and was considered as the "adults" in the whole organization. It's definitely much more organized than DOS/Win 3.X back then.

jaen• 5 months ago

I'm very curious about what you consider a real "design" then?!

The Windows NT kernel had a better design than most contemporary and succeeding operating systems (including the various Unixes, Linux, BSD, plan9 etc.).

Modern kernel designs like Google's Fuchsia have more in common with NT than POSIX/Unix/Linux.

In particular, the NT object manager approach subsumes the Unix "everything is a file, well, not quite, oh... uhh.. let us slowly fix that by adding stuff like signalfd, memfd, pidfd etc. ahh hmm, these still do not exactly fit into a FS mold... ah crap, still missing a proper ptrace FS analogue" design approach that eg. Linux has taken in the last two decades.

It also had powerful primitives like eg. APCs that could be used to build higher-level kernel features elegantly.

canucker20162• 5 months ago

Looking at the authours of the design workbook, I note that there's a college intern in the list.

Who says college interns don't do real work?

I remember when he told several other interns that they had got "Hello World" to run on the nascent Windows NT.

Windows NT/OS2 Design Workbook

Related Stories