Why is a registry file called a “hive”? (2003)

https://devblogs.microsoft.com/oldnewthing/20030808-00/?p=42943

by tapoxi • 4 years ago

88 104 4 years ago

devblogs.microsoft.com

dekhn• 4 years ago

because it's full of scum and villany.

wnevets• 4 years ago

I don't get it, is that something hives are known for?

adrianpike• 4 years ago

It's one of the more famous quotes from Star Wars: https://www.youtube.com/watch?v=Xcb4_QwP6fE

CurtHagenlocher• 4 years ago

Only the wretched ones.

(Hint: use a search engine.)

zabzonk2• 4 years ago

Oposite I would have thought - full of nice little humming creatures, making delicious sweet stuff.

nescioquid• 4 years ago

You must be young.

Pxtl4• 4 years ago

Better question: why aren't there comments? It's 2022 and there are still no comments in regedit?

userbinator• 4 years ago

You can make a string value with an unused name and use that as a comment, if that's what you're referring to?

Pxtl4• 4 years ago

This idea is also the recommended workaround for JSON, and it is equally wrong there. For one thing, then you can only comment the key/folder/object, not the individual value.

indrora• 4 years ago

For something that wasn't meant to be interacted with by humans on the regular, I can see why you wouldn't include comments.

umanwizard• 4 years ago

Classic mistake. The creators of JSON thought the same thing, and that decision is here forever to annoy us.

assttoasstmgr• 4 years ago

Because the registry is a database. Why would you put comments in a database?

Regedit is a table editor of that database. So again, would you insert random records into a database as comments?

Now if you're asking "why aren't there comments in .REG files" the answer is there are, prefix it with a semicolon (not well documented):

https://docs.microsoft.com/en-us/previous-versions/windows/e...

Probably wouldn't have been difficult to guess though, since they follow the .INI file syntax (which ironically is now used by SystemD)

Pxtl4• 4 years ago

The registry is as much a database as it is a very large config file, and config files, as you've rightly observed, have comments.

dragonwriter• 4 years ago

Because the registry is a database. Why would you put comments in a database?

A database stores whatever data is necessary for the application. If comments for application records are needed, that includes comments.

A self-documenting registry where applications registered user information about keys that they used (whether or not the application populated them) as well as apps populating data for keys, so that navigating to a key in the UI would provide you with the documentation from any app or apps concerned with that key as well as the ability to review and edit the data value would be awesome.

chasil• 4 years ago

Oracle has user/all/dba_tab_comments and user/all/dba_col_comments.

The only places in my legacy databases where these are populated emerged from the Erwin data modeler. We are lazy in not populating them with new tables.

I wonder if SQLite has something similar.

zinekeller3• 4 years ago

Backwards compatibility. I'm not remotely kidding here. There are still programs that directly parse these files instead of using Windows' APIs and the format itself is fragile, and in the 90's the concern was more of disk speed access (where comments will slow down reading of registry files) meant that the designer have opted to not put comments (as you expect from that time that the manual is literally a book or a separate CHM or TXT file, with user comments stored at the "Notes" part of the book or another separate TXT file). It's similar to the binary DOC format pre-Word 97 which includes actual C pointers (so it's less of a document format and more of a snapshot of Microsoft Word memory).

Edit: Some have noted Group Policy and its note field. I've disqualified that because it's not readily available in Home versions of Windows, because third-party applications are often not bundled with the necessary ADM/ADMX files necessary (which you might also want to comment), and the fact that the parent commenter disqualified the text-based REG files (which supports comments).

ChrisSD• 4 years ago

The registry is the raw database. The Policy Editor is the human interface with comments and other niceties.

Lammy• 4 years ago

This has sort of existed since Windows NT 4.0 days in the form of Administrative Templates, with the newer "ADMX" format shipping with Windows Vista / Server 2008:

https://docs.microsoft.com/en-us/previous-versions/windows/d...

https://admx.help/

https://jeffpar.github.io/kbarchive/kb/225/Q225087/

kazinator4• 4 years ago

What stops you from creating a string key called Comment? :)

tinus_hn3• 4 years ago

Because you can’t automatically manage a configuration database that has comments. If the applications can remove and recreate their registry values and keys there is no way to keep comments associated with these things.

ComputerGuru• 4 years ago

That’s funny. If I loved bees and was obsessed with them, I’d have done the same thing this bee-hating developer did.

romanhn• 4 years ago

What makes it funnier is that the registry developer is not the same person as the bee hater, if I'm reading this correctly.

pkrumins• 4 years ago

If you think about it, the Windows registry is the first and most deployed NoSQL database.

loudgas• 4 years ago

Not really. Databases predate SQL.

throwawayboise• 4 years ago

And they predate Windows.

Const-me3• 4 years ago

Each Windows installation only has 1 registry, but it normally has multiple ESENT databases https://en.wikipedia.org/wiki/Extensible_Storage_Engine

These databases are NoSQL as well. Albeit somewhat closer to traditional SQL than the registry: ESENT databases are made of B-trees, have column schema, primary/secondary indices, and support transactions. Unlike SQL servers, ESENT doesn’t have relations across tables, doesn’t have a query language (instead it has a low-level API to move cursors), and doesn’t support remote nor inter-process access, a DB is exclusive to a single process.

zinekeller3• 4 years ago

Each Windows installation only has 1 registry

This is clearly false, how could you store user-specific data? (Hint: it's the NTUSER.DAT on each of the users' directory.) Additionally, you could create binary registry files and load them, funnily in much the same manner as how would mount drives on Unix and Unix-derived/inspired systems. Unless you consider that as a single root, the comparison that "Windows only has 1 registry" (which is wrong since that system-wide registry is backed up every boot and shutdown) is silly.

banana_giraffe• 4 years ago

Wouldn't something like Berkeley DB be closer to having that honor?

tsuru• 4 years ago

I'd wager MUMPS is https://en.wikipedia.org/wiki/MUMPS

meepmorp• 4 years ago

Mainframe databases from the 60s.

zdw• 4 years ago

Honestly, the registry as an idea is a great concept - structured, typed data to define the configuration of an entire system, which all applications use for config? Remind anyone of YAML loaded into K8s?

somat• 4 years ago

It is a great concept, it makes sense from any reasonable viewpoint, yet for some reason I loath the registry with a passion, not exactly sure why, but I think it comes down to the question.

Why do we need an additional tree to store configuration?

FridgeSeal2• 4 years ago

I think because it’s used haphazardly and applications make messes that they never clean up. Also, having both env vars, local config and registry creates sort of a smorgasbord-stuff gets thrown everywhere.

If all you were allowed for config was the registry,and application scopes were more tightly defined and enforced, it would probably be a lot nicer.

zmmmmm• 4 years ago

I always just found it incredibly hostile to access and use. Hate to think the hours I've lost from my life using Regedit.

But that goes for all sysadmin stuff in Windows. Give me plain text files I can grep and edit with vim, backup by making copies, diff, version etc. Make it JSON if you want but let me use real tools and not the horror show of Regedit and it's ilk.

pie_flavor3• 4 years ago

You don't. You can store it anywhere else you like and most programs do. The system uses it to store its own configuration so other programs can configure the way they integrate with the system.

andrewstuart• 4 years ago

The Registry sounds like a great concept but it's the worst architectural decision Microsoft ever made for Windows.

The idea was to move away from text based ini files, which is previously how Windows apps were configured. At the time I thought "awesome".

However the registry is just a black box full of binary garbage, chewing gum and rubber bands that tightly couples everything in Windows.and it's like spaghetti and can't possibly be unwound.

.ini files were a much better idea, in hindsight.

I've always wished Microsoft would have the courage to somehow get rid of the registry but I don't think it will ever happen.

I feel like Windows needs a clean start that implements its modern APIs, ditches its commitment to backward compatibility, gets rid of the registry, and makes everything including the UI consistent and modular. That's a Windows worth getting excited about.

The unix model of plain text files for configuration is a much better idea than a Registry.

Without the Registry there wouldn't be the concept of "Registry cleaners" https://en.wikipedia.org/wiki/Registry_cleaner <--- read this link to get a sense of all the bad things about the Registry.

The Registry is the reason why you need an installer instead of being able to drop application files into a directory and run them.

Sakos4• 4 years ago

Also if the registry is corrupted in any way, there's no way to recover it. It's crazy that there's this single point of failure.

dredmorbius4• 4 years ago

The Registry cannot be documented.

Configuration files can be.

Configuration files can be individually versioned and managed.

Configuration files can exist in multiple instances, be swapped out or referenced for a specific invocation.

I could go on.

Yes, the Registry is an absolute disaster.

pie_flavor3• 4 years ago

The Registry is the reason why you need an installer instead of being able to drop application files into a directory and run them.

You can drop a program into a directory and run it. If you want it to integrate with the OS or DE, you need an installer to configure it in the centralized location. No difference there between Windows and Linux.

Furthermore, the specific aspect of it you reference (that it is binary instead of textual) has almost nothing to do with the problems you say are caused by it (that install scripts need to add keys to it, or that obsolete keys are retained and third party scripts clean them up with heuristics). None of that would change if it was INI. A binary configuration format has nothing to do with the architecture of what it is used to configure.

If it was truly a terrible decision then they would surely have not recreated it with Windows Phone, which had no win32 components that would have depended on it and no win32 API compatibility at all. But they did.

I suspect the reason non-Windows people don't like it is just that it is a pure expression of Microsoft's API-oriented (as opposed to file-oriented) approach to Windows, which does not mesh with the *nix mindset.

baisq3• 4 years ago

I feel like Windows needs a clean start that implements its modern APIs, ditches its commitment to backward compatibility, gets rid of the registry, and makes everything including the UI consistent and modular. That's a Windows worth getting excited about.

A version of Windows that can’t run most of my favourite programs and games is something I definitely wouldn’t be excited about. If I wanted a computer that can run no software that I care about I would buy a Mac.

flomo• 4 years ago

I recall when a Linux kernel dev broke it down like 1) files can be slow and correct, 2) or files can be fast and incorrect, 3) but if you want fast and correct, you need a database. Slashdot and other tabloids reported this as "Linux developer says Linux needs a Registry". (In the meanwhile a lot of devs have come around with sqlite.)

jahewson• 4 years ago

I always wished I could just mount the registry as a file system. Best of both worlds?

ogurechny3• 4 years ago

“Registry cleaners” have been irrelevant for 2 decades, and have been a sure sign of a scam for most of that time. Even in the '90s, the main thing they did was exporting the whole tree into one giant text file, then deleting/moving the old database and importing the dump into a fresh copy in DOS shell before system boot (reg.exe was/is a combined DOS+Windows executable). You could do the same manually. The size difference could be significant because file allocation scheme was not good enough, and slowly inflated it with use.

How program stored its own data made little difference. It could be a configuration file, it could be a designated place in registry — as long as nothing else touched it, everything was fine. The core problem was with the system configuration files. Each program, old and new, had to understand all kinds of entries that could be added there by each other, and that included would-be future versions. Given something like

[ZZZ386]

...

DRIVER=MSSTD.SYS,EXTRA.SYS

...

every programmer had to know that there could be more than one driver (not really obvious if all common machines around you worked with single standard one), that comma was a delimiter, that the order could be important (and therefore you couldn't always add your entry to the end). But they didn't expect any of it, so programs broke themselves, or silently broke other programs. Registry was introduced as a system-wide black box with a strict API. That allowed correctness and security checks (at least on NT), adding compatibility hooks for specific programs, and so on.

However, it was not a sandbox of any kind. Any change in system configuration that affects more than just your program would still require installer and uninstaller logic. You probably imagine something similar to external manifest files that would hold, say, program-related CLSIDs. Now imagine that enumerating CLSIDs requires enumerating all manifest files in the system… #8-0

I suspect it has been explained more thoroughly somewhere on the same linked blog.

tinus_hn3• 4 years ago

The Registry sounds like a great concept but it's the worst architectural decision Microsoft ever made for Windows.

The idea is great, the problem is with the implementation. You can’t have a configuration database so central to your system and then implement it using a fragile database engine that breaks itself all the time.

a-dub• 4 years ago

would be funny if someone wrote a comedy winfsp driver that exposed the registry as a gigantic win.ini file.

kazinator4• 4 years ago

At around the turn of the millennium, I ported a COM object from Windows to Linux. I made everything required work, from cruft DllMain and DllCanUnloadNow to

CoCreateInstance and whatnot. The COM object configured itself using the Registry, so I ported all the API functions. I wrote equivalents that worked with a text-based representation. It was great: the code thought it was using the real registry, but you could go in there with "grep -r" and your editor to muck around in it.

HKEY_LOCAL_MACHINE mapped under /etc, HKEY_CURRENT_USER under $HOME/.whatever type of thing.

Jaruzel• 4 years ago

The main problem with the Windows Registry, is that it doesn't clean up after itself... Programs are unlikely to fully purge their settings, and even Windows leaves detritus everywhere (just look in the USB enumeration keys - EVERYTHING you've ever plugged in to USB is still listed there).

skissane2• 4 years ago

The Registry sounds like a great concept but it's the worst architectural decision Microsoft ever made for Windows.

I think the problem wasn't so much the idea itself, it is just that the implementation left a lot to be desired. A better implementation of the same idea could have been much more successful.

The registry could have come with a schema, whereby each key has a class defining what values it is allowed to have. I think Microsoft may have actually had some plans to do that, because every registry key has a "class" (lpClass parameter of RegCreateKeyExW) – however, few registry keys have the class field set, and almost no software pays any attention to it – suggesting that whatever Microsoft was planning with respect to a registry schema never came to pass. There could have been a "schema hive" containing class/attribute definitions, and then any attempt to set a key/value on a schema-enforced key would be validated against the schema hive.

If keys had an enforced schema, the registry could be self-documenting – the class and attribute definitions could have a "description" value explaining what each is for. I suppose Microsoft couldn't force third parties to write meaningful descriptions, but they could have set a positive example. They could also have had a tool to auto-generate schema documentation (in HTML or RTF or whatever) from the descriptions in the registry schema.

They could have implemented something like a "foreign key" – a link type which will not let you delete the target while other keys have values pointing to it. (Not the existing REG_LINK which is much more limited; REG_SMART_LINK maybe?) This would likely need some kind of "backlink" index for fast retrieval of back-references. Maybe each key could have a "virtual attribute" which contains a list (REG_SMART_LINK_LIST) of backlinks. The schema could specify what are the allowed target types for any link attribute.

They could have a package database in the registry, and enforce that every registry key is owned by a package. Uninstalling a package automatically removes all the keys it owns.

They could have shipped with an IFS which exposes the registry as a pseudo-filesystem, which would have made it easier to manipulate. (Cygwin kind of does this with /proc/registry, and PowerShell has something similar, but it would have been better as a proper filesystem which appears like any other.)

It could even expose the registry data as JSON or XML files, dynamically generated at runtime. \\registry\HKEY_LOCAL_MACHINE\Foo\Bar\Baz\__REGDATA__.json could be a JSON file containing the values of the HKEY_LOCAL_MACHINE\Foo\Bar\Baz registry key. If you write to that file, it could parse the JSON you write and update the values appropriately.

The unix model of plain text files for configuration is a much better idea than a Registry.

I actually don't like that model. You end up with text files littered all over the place. Automating configuration is a mess because every app has a different config syntax. It is a flawed model too, just with a different set of flaws.

The Registry is the reason why you need an installer instead of being able to drop application files into a directory and run them.

Not really. An application can be designed to not require any registry keys, and so if all its registry keys are missing it just uses sensible defaults. Or an application can be defined to check for its required keys at every startup and create them if they are missing.

Microsoft could have defined a way for an application to embed a small registry hive in its EXE. When launching the EXE, it would check if those keys exist in the registry, and automatically load them.

In Microsoft's defence, the Registry was designed in the early 1990s (it first shipped in Windows 3.1 in 1992), and a lot of the ideas I am proposing may have had too high a performance cost for the limited machines of the early 1990s, or relied on technologies or ideas which people hadn't thought of yet.

jiggawatts• 4 years ago

The concept of the registry is just fine. The implementation had a few small architectural issues, but throwing out the concept because of a single specific implementation is a mistake.

Text files have MORE problems, not less. Your familiarity with them and your unfamiliarity with the registry doesn't make text files better, it makes them more familiar. Similarly, even if you know the Windows Registry well, that's the Windows-specific implementation that dates back to the 1990s!

If the registry concept was re-done today, with all of the lessons learnt across decades of software pattern evolution, it would be done differently.

What you probably don't realise is that Registry provides an awful lot that prevents every developer having to reinvent in a Unique and Special Way that is incompatible with everything else.

It gets "right" the following:

- Hierarchical storage of key/value data.

- A single consistent GUI and API across all application settings.

- Permissions on a per folder path and/or per key basis.

- Integrated audit logging for all access, including both read and write access, down to the individual key level.

- Binary (byte array) data stored natively.

- Strong typing for 32-bit and 64-bit integers.

- Special string types such as "substitute environment variables in this string" vs "just a plain string".

- Layering of settings. E.g.: Machine, User, Default, etc...

- Integration with the Group Policy engine, where there are "policy enforced" and "default preferences" that can be pushed out centrally.

That last one needs expanding, because Linux has NOTHING like it, and alternatives don't even come close. The policy engine is declarative, with a "template" file specifying the keys and allowed values. This has matching multi-lingual files so that administrators that don't speak English can see the names and multi-paragraph descriptions in their native language. The policy settings apply and un-apply in layers automatically. Moving locations can automatically deploy a setting, and then wind it back if you return to your original location. Thousands of settings can be centrally administered by one admin for tens of thousands of machines with a few mouse clicks. Linux config files are a single layer and are effectively write-only. You can't read them with an API or merge/unmerge changes on the fly. There is just no way to solve this problem with general text-files.

If I were to reinvent registry files now, in 2022, I would just fix a few implementation specific annoyances:

- More strong types. E.g.: network addresses, DNS names, GUIDs, etc...

- The "layering" of Group Policy is manually implemented in app code, so most apps outside of the Enterprise space don't have policy integration. I would make the policy settings automatically overlap the same key names so that they're merged by the "registry engine" instead of the app.

- Writing arbitrary values into a single share hive is a mistake. I would create a hive-per-application.

- Requiring kernel transitions for registry key reads is also a mistake. The "user" hives belonging to individual applications ought to be accessed directly by the associated app processes. I've seen hacks that implement this using API rewrites that improve the performance of some business applications by 20-40%!

- Registry hives should have "layers" much like Docker files in the physical sense, not just the logical sense. So for example an application patch would be a new delta layer over the base layer, but then policy would always layer on top of the latest patch, whatever that may be.

- Stronger tooling for developers. Make policy template files trivial to create. The template files should auto-generate matching strongly typed configuration objects in multiple languages, and these should also integrate with CLI command parsers, environment variable overrides, etc...

- Use a more standard base storage format, such as SQLite.

- Make it cross-platform and open-source.

- Similarly, the policy engine shouldn't be so closely tied to a single product (Active Directory), but should be a pluggable module that large-scale enterprise software can also reuse as an internal component. Think Citrix, Anti-Virus products, Backup, etc... anything that needs to manage hundreds of servers from a console with policy, preferences, etc...

dtheodor• 4 years ago

Would you say that a database is a "black box full of binary garbage, chewing gum and rubber bands that tightly couples everything in Windows.and it's like spaghetti and can't possibly be unwound"?

I don't think you would, that's not the conventional wisdom. Databases are considered to be the epitomy of clean, reliable, and efficient representations of data. It is text files that lack these capabilities and are considered a non-standard mess when it comes to data.

The registry is just a database. It may have not seen the best usage it could have from windows and applications, does that make it a bad architectural decision? Would we think different on the registry if it was just SQLite (or would we in this case think differently of SQLite)? Personally I firmly believe that the registry approach is a great architectural decision, for all the reasons outlined here: https://sqlite.org/appfileformat.html

eternityforest6• 4 years ago

I implemented a registry once, where plugins in an app could store data. It was kind of horrific.

Text files are great for config because you can version them and they tend to be somewhat self documenting. And with no registry you can use very useful patterns like one file per object.

One of Linux's nastiest things, that is mostly fixed now, was things like hosts files and fstab, individual files responsible for multiple objects that are likely to be "owned" by different packages. Now most things have a config.d folder or something so you can use multiple files.

Registries take away the ability to represent things using the file hierarchy and manipulate it with common tools.

Stampo002• 4 years ago

I wish I'd thought of that! I used to work with someone who was unbearable and he also hated butterflies. I should have snuck as many butterfly references in the code as I could.

You know who you are!

imron• 4 years ago

Honestly speaking, if you're the type of developer who would do such a thing, there's a high chance that you are the unbearable one.

Stampo002• 4 years ago

In reality, instead of trolling this guy... I just left the team so I didn't have to work with him anymore. Which is the correct, mature, professional thing to do.

But my inner child still likes to dream. As far as revenge fantasies go, it's pretty tame, don't you think?

JoyfulTurkey• 4 years ago

I had a feeling this was going to be a Raymond Chen post. Loved reading the Old New Thing back in the day. Amazingly it looks like it is still going strong.

ogurechny3• 4 years ago

0 comments

This is a blatant lie, I know posts on Old New Thing with zero comments have been extremely rare. And of course:

https://web.archive.org/web/20091231092140/http://blogs.msdn...

Note that pingbacks were also available, and some posts had quite a lot of them from various years, creating a web of somehow related pieces of information. After 2012 the comments were loaded with AJAX, its endpoint obviously stopped existing, and it wouldn't work on Web Archive anyway. However, these resurfaced in 2016 after another move. And someone on bytepointer.com has created a sane static copy which deserves an applause.

Also, various old custom text style markup was broken in other posts, but it's a general problem with all MSDN content. People doing the conversion just didn't care. I guess informing them that some old knowledge base articles were actually distributed as self-extracting archives (IIRC, in custom format, so you have to run them in an emulator to unpack) with text and code files (in DOS code page, with occasional pseudographics) would make them panic and run away.

So there's a corporation the size of Microsoft, and one of the most highly regarded places to set “correct” approach to Windows programming, and the most they can offer to Raymond Chen is “Hey, old man, our engine supports bold and italic, and comments from Microsoft® Account™ logins, you can import your posts and fix them manually. Next!” The announce post had an excuse (personal data protection laws)

https://devblogs.microsoft.com/oldnewthing/20190313-01/?p=10...

HOWEVER, it does not cover pingbacks (at least pingbacks from oldnewthing to oldnewthing itself). Also, even in 2003 we see the remains of “old” old comment system, so at least some comments had been impossible to delete all along even before the move. Tsk.

/grumble

veltas• 4 years ago

At some point MSDN replaced the names in comments with email addresses associated with accounts. My email address wasn't public but all of a sudden it was, at Microsoft's discretion! I complained about this, potentially this was the data concern they had, because it was actually pretty dumb on their part.

iggldiggl• 4 years ago

And someone on bytepointer.com has created a sane static copy which deserves an applause.

Oh wow, nice, thanks for mentioning that. I've been grumbling about the lost comments, too…

gjvc3• 4 years ago

this is a good example of why whimsy is not to be tolerated when creating foundational building blocks

kazinator4• 4 years ago

No it isn't; the hive metaphor entirely fits the Windows Registry, which is a kind of hornet's nest where configuration goes to disappear.

The person who came up with that was developing it seriously, and the naming almost certainly had to have "buy in" from other "stakeholders" in the project.

gjvc3• 4 years ago

seems like you didn't read the actual article. it's short enough to include here.

""" Because one of the original developers of Windows NT hated bees. So the developer who was responsible for the registry snuck in as many bee references as he could. A registry file is called a “hive”, and registry data are stored in “cells”, which is what honeycombs are made of. """

agumonkey• 4 years ago

https://devblogs.microsoft.com/oldnewthing/20030814-00/?p=42...

His blog is full of nice reads

Why is a registry file called a “hive”? (2003)

Related Stories