emily-c

In many ways NT was a new, ground up implementation of “VMS NT”.

Most definitely. There was a lot of design cleanup from VMS (e.g. fork processes -> DPCs, removing global PTEs and balance slots, etc), optimizations (converging VMS's parallel array structure of the PFN database into one), and simplification (NT's Io subsystem with the "bring your own thread" model, removing P1 space, and much more). SMP was also designed into NT from the beginning. You can start seeing the start of these ideas in the MICA design documents but their implementation in C instead of Pillar (variant of Pascal designed for Mica) in NT was definitely the right thing at the time.

Before VMS there was the family of RSX-11 operating systems which also had ASTs (now called APCs in NT parlance), IRPs, etc. Dave Cutler led the RSX-11M variant which significantly influenced VMS. The various concepts and design styles of the DEC family of operating systems that culminated in NT goes back to the 1960s.

It's sad that the article didn't mention VMS or MICA since NT didn't magically appear out of the void two years after Microsoft hired the NT team. MICA was being designed for years at DEC West as part of the PRISM project.

I don’t think anyone has ever bothered to make FS code async, it’s already quite horrible as it is. Not in a mainstream kernel anyway. [...] Same for Windows disk IO with IOCPs.

IOCP merely allows IO completion to happen on worker threads in a threadpool rather than the particular thread that initiated the IO request. Many concepts in the NT kernel come from a long line of production kernels in mainstream DEC operating systems going back to RSX-11M and VMS. That entire lineage of operating systems (culminating in NT) all have true asynchronous IO using IO Request Packets (IRPs) within the kernel to initiate/queue IO requests and immediately return. There are individual cases within NTFS where blocking will occur but those are special cases rather than the rule as the kernel IO system in general is entirely async.

I hope this is a joke.

"Embedded software" and "embedded system" also have similar issues with being nebulously defined in various situations. Some things will never perfect and at least firmware is kind of cute.

Usually this is still host firmware and not a secondary controller, at least on x86 platforms. To detect/use the USB controller you still need to configure the chipset/root complex and do an initial PCIe bus scan to set up PCIe BAR apertures. After that occurs you need a (primitive) USB stack that is able to talk to the USB controller to enumerate the USB devices as well as block storage and filesystem layers. All of this code is implemented as a collection of DXE drivers that implement various UEFI protocols which are initialized in the UEFI DXE phase that runs after SEC and PEI phases. On Intel platforms PEI does things like training the DRAM and PCIe links so memory is always available to DXE. Unfortunately, there's still a lot of code that needs to run to get this to work.

After FW update binaries are located it's not uncommon to write them to a scratch flash and then reset the system. On reset somewhere in the flow the scratch flash is checked for an update and then the hardware sequenced flashing registers in the chipset are utilized to actually flash the firmware. Another reset is performed to boot from the freshly flashed firmware.

There are variations on this flow depending on the firmware implementation and platform/vendor which can simplify it but that is usually the basic idea. Various microcontrollers are definitely employed for other platforms (even on x86, such as the embedded controller though these perform auxiliary tasks to the host firmware rather than the whole thing).

Updating without a CPU installed usually is on the embedded controller itself but that's not a normal update flow IIRC.

Recording of the opening: https://www.youtube.com/watch?v=Dfd6J3M-FSk

Thanks :) I wish there was a consolidated list of what kinds of information gets measured that is easy to point to.

Unfortunately, I don't believe there is any up to date and detailed public documentation on the modern DRTM flows that exist on both Intel and AMD platforms. Maybe documentation has been recently updated but I’m not sure if I’m able to share more beyond what I already have.

DRTM does not remove any malicious firmware provided code or data. Traditionally it is merely a measurement mechanism that happens after ExitBootServices which measures platform state in an unforgable manner. Practically the DRTM event can also cause certain chipset registers to get locked or SMM supervisors to get launched depending on the platform. SMM and ACPI tables (on some x86 platforms certain tables are rebuilt) are measured into a PCR by the secure loader or security processor during the DRTM event. The idea is that if malicious code or data was present then the PCR values wouldn't match the previous boot session and TPM secrets wouldn't unseal.

While what you said is technically correct, it is by design and any compromised firmware can do as it pleases before the DRTM event at the cost of getting caught and having the device fail attestation or not be able to access encrypted data (depending on what policy is layered on top of DRTM itself as it is just a security primitive). By having PCRs get reset during the DRTM event secrets are much more reliably able to be sealed to specific PCR values.

Did we mention that a TPM isn’t going to protect you from UEFI malware that was planted on the device by a rogue agent at manufacture time?

DRTM, a technology supported by Windows 11 that is layered on top of the TPM, aims to solve this very problem.

I am talking about the platform more than the ISA. Some interesting resources:

https://sites.google.com/site/pinczakko/system-address-map-i...

https://www.youtube.com/playlist?list=PLBTQvUDSl81dTG_5Uk2my...

http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tu...

http://xillybus.com/tutorials/pci-express-tlp-pcie-primer-tu...

https://www.amazon.com/Quick-Boot-Pete-Dice/dp/1501515381/

https://opensecuritytraining.info/IntroBIOS.html

https://acpica.org/sites/acpica/files/asl_tutorial_v20190625...

And of course the MindShare books.

I'm not sure why you have legacy in quotes, that is what it is. Modern platforms do not act like a PC AT and there is a ton of cruft to keep those interfaces working which is set up and facilitated by firmware (not so open, at least TianoCore exists regardless of your thoughts on UEFI). Modern x86 systems programming is so much more than these interfaces. I think that it would be beneficial for others to understand how things are actually working on modern platforms.

Regarding your linked post:

For corporate users legacy methods of booting /are/ a security threat. Firmware attacks are a real threat for enterprises. Having IBVs all hand roll things like the S3 boot script or communication with SMM (not using standardized com buffers) is not wise. Secure boot combined with modern security features like using DRTM to remove firmware from the trusted computing base are important and should not be discounted because it's not an 40 year old interface. The PC platform is historically extremely insecure and there is a ton of work still to be done to effectively provide the primitives needed to secure the platform against determined attackers.

If you want to have a USB stack in SMM owning your USB controller to emulate port 60h with IO SMI traps that's fine but it's not a reason to hold the platform back.

These are good examples and this might be slightly off topic but I wish that there were more resources on how modern x86 platforms actually work instead of using these legacy interfaces that are either emulated in SMM with IO port trapping or relegated to requiring chipset hacks to keep them working (such as VGA device hacks in the PCI spec or well known chipset legacy IO ports decoding to the LPC bridge).

While it is essentially impossible to truly understand every aspect of todays platforms I think that one can build a lot of intuition about how things work by studying how transaction routing is handled in the platform, e.g. how the uncore and chipset routes TLPs (whether it's raw PCIe or packets essentially wrapped by other interconnect protocols like IF/HyperTransport/DMI/QPI/etc) through the system with various base address registers and decoding windows. This gives way to understanding how modern MSI interrupts and DMA works as well. While I understand that truly going into things like PCI configuration, ACPI, UEFI, chipset interfaces on bus 0, modern booting, DMA and interrupt remapping, DRAM, register locking and security, security processors, power management, various types of runtime firmware, etc is a a many-years-long exercise I think that the core ideas don't need to be spread around all over the place for one to get familiarity with the ideas and patterns that underpin our PCs.

I think that a general conceptual overview on which one can hang more detailed knowledge acquired by reading specs is needed with the amount of complexity that exists today. There are some really good resources that I've collected over the years but there is still something missing. Maybe I or someone else will get around to it some day.

A lot of critical system configuration is stored in the SYSTEM hive which isn't explicitly backed by a file mapping and is loaded at boot via firmware services so this will be fast as it is non-paged and mapped in kernel space for the entire boot session. On newer builds of Windows 10 other hives are memory mapped into the usermode address space of the minimal Registry process. Whenever you do a registry read the kernel will temporarily attach your thread to the Registry process' address space and the read to the UM mapped section will occur which will naturally fault in the data from disk. The requesting process' thread will then be unattached and the information will be returned. Since non-SYSTEM/ELAM hives are memory mapped the kernel's cache manager and memory manager subsystems are the ones that "own" and control the mapped memory. The file cache is tuned based on the particular system's hardware characteristics to be as performant as possible. There are registry-specific caches in between to reduce the need to attach to the Registry process but this isn't going to be a disk IO speed bottleneck.

The program you're thinking of is procmon which is a part of the Sysinternals suite of tools.

This is awesome. It’s always nice to see new Win32 productivity software. Thank you!

This is correct. The functionality of LOADALL now largely happens in the RSM instruction which returns back to normal execution context from an SMI. In modern firmware, before the SMI handler is invoked by the SMM dispatcher the firmware will switch the mode away from real mode and have its own GDT/descriptors. All this needs to be restored on SMI return thus RSM will parse a specially formatted block of processor context to dump everything back.

Most legacy code is this way unfortunately. The actual techniques employed by the "called out" code are not uncommon. I don't think that people are defending the lack of clarity but rather what the code is doing itself. It could be written more clearly, though one could argue that the lengthy series of articles does just that. Maybe it's an exercise for you to write it how you want.

Have you taken a look at some major C codebases? It's not justification but an appeal to authority seems pretty unnecessary :)

You see similar code to his unconditional executeNoMatch at the end of the table in table based code all the time. Additionally forEachObject is a macro that immediately initializes a variable. For projects like this the code is completely fine. I'm not sure what is particularly dangerous about this code for a small to medium size project.

I second every book recommendation here. Another great one is The Standard C Library.

This is still pretty common for things like self extracting archives on Windows.

  copy /b "%windir%\system32\extrac32.exe"+inputarchive.cab selfextract.exe

It is still possible but is BIOS dependent :) With UEFI, graphics is preferred to be done with the GOP driver/protocol (which will ultimately interact with video BIOS/graphics card option ROM for you). During early boot setting up the VGA decoding can either not be done or disabled when booting with UEFI. You might need your UEFI BIOS to be in CSM mode (legacy BIOS compatibility mode for UEFI) to get this to work and a lot of modern systems don't support CSM as of late. You can write to the framebuffer using GOP runtime services, and the VGA IO ports would likely do nothing since they wouldn't be decoded to the graphics card. Full fledged OSes just load a graphics driver and directly talk to the hardware -- something that is unfortunately not wieldy for hobby OSes :(

You're still relying pretty heavily on the BIOS to write to this region that you otherwise wouldn't be. Getting the legacy framebuffer region set up to be decoded to the graphics card is using some hacks to get it to work over PCI[e]. The memory controller/PCI root complex needs to be configured to forward transactions to the the legacy A and B segments to the PCI bus using a proprietary bit to enable that MMIO hole. To enable correct decoding of that, all PCI bridges on the path to the graphics device need to be specially marked in their config space bridge control as being "VGA compatible" so that legacy framebuffer hole requests and VGA IO ports can ultimately be forwarded to the graphics device. This needs to exist because the root ports/host bridges in the system base and limit need to encompass the addresses of their children as well as needing to exist in a hole that the memory controller/root complex knows to forward requests down to PCI.

You use WinDbg and do kernel debugging. The newer WinDbgX UI is quite nice lately. However, there's no replacement for live debugging on the same machine the way that SoftICE did it.

It's important to realize that you are not your code. Linus' reply comes off as passionate but he clearly cares enough to write such a detailed response.

You can thank Jay Miner for that. The Atari is sort of like an Amiga-lite in a lot of respects.

This book is great. I would say that it is for beginners as long as they have some semblance of C understanding and spent a few hours (days?) reading up about assembly and machine code topics. These skills take years to truly master and the book starts from the beginning. I think the difficulty of asm is overrated (its not uncommon for young kids to learn this sort of stuff).

I think one of the greatest initial hurdles of being a good reverser is not being afraid to wade through large amounts of disassembly and being comfortable with not understanding whats going on for long periods of time.

The CSME is on the PCH[1] (when one exists). (However AMD elected to put their PSP on-die as an IP-block). It is the root of trust of the system and takes the CPU out of reset even though it's not on the CPU itself. x86-based machines have pretty much always had multiple auxiliary processors/SoCs that control various aspects of the system such as the embedded controller (a direct descendant of the 8042 keyboard controller). Another example of this is the AMD SMU which lives on the chipset and coexists with the PSP (the talk in the link gets code running on this as you described)[2]. Interestingly the Chromebook EC firmware is open source[3]. Joanna Rutkowska has done a nice overview of some of the security aspects of these topics as well[4].

Ideally there would be a standard way to verify and measure all the various firmware images (and potentially option ROMs which would be theoretically measured with SRTM via UEFI) for these system processors during boot during a DRTM event (with Intel's STM/AMD SMM supervisor enabled as well) but there is a long way to go for that.

[1] https://i.blackhat.com/USA-19/Wednesday/us-19-Hasarfaty-Behi...

[2] https://www.youtube.com/watch?v=iYvhHey_dTk

[3] https://chromium.googlesource.com/chromiumos/platform/ec/+/m...

[4] https://www.youtube.com/watch?v=rcwngbUrZNg

This is awesome. If you haven't seen this talk about the AGC you should definitely check it out: https://www.youtube.com/watch?v=xx7Lfh5SKUQ

These types of resources are awesome though I've always wished that they at least briefly touched upon the x86 memory model/consistency. Understanding the concept of memory barriers should be considered fundamental.