Nix 1.0 released: purely functional linux package manager

http://hydra.nixos.org/build/2609700/download/1/manual/#ssec-relnotes-1.0

by moonboots • 14 years ago

120 59 14 years ago

hydra.nixos.org

_seininn• 14 years ago

I may be naive in asking this, but WHY? Why is being "purely functional" an advantage for package managers?

gizmogwai2• 14 years ago

Well, the obvious answer is that it is an advantage for the same reason that being purely functional is an advantage in other situations. More specifically, the benefit of having no side effect in the package management is that it greatly simplifies the management of multiple versions of a given package. It also simplifies the rollback of an unwanted operation in most cases.

stephenjudkins• 14 years ago

http://nixos.org/nix/ summarizes some of the advantages of this approach. To summarize, it promises to be fundamentally safer and more flexible than traditional package managers. Package management, IMO, is an area where a "purely functional" approach could really shine, since correctness and replicability are so important.

Cieplak• 14 years ago

Can anyone explain advantages this might have over, say, Synaptic?

gizmogwai2• 14 years ago

As far as I know, Synaptic is not able to handle multiple versions of a given package simultaneously.

jerf3• 14 years ago

I've been eying this as a replacement for Gentoo. I like it well enough for the configurability, but I tire of the way I'm rolling the dice every time I update a package. (Binary distros like Ubuntu have this same problem, just less so.) The primary use case for me would be enabling me to stay cutting edge on software, while making distro upgrades reversible. If my new kernel blew up my system, I don't go scrounging around for boot disks, I pick the previous image on next boot.

I don't know of any other distro that can pull that use case off right now. Or any other OS, really.

darkstalker• 14 years ago

I don't see how this could be better than portage

mappu2• 14 years ago

    If my new kernel blew up my system, I don't go scrounging around for boot disks, I pick the previous image on next boot.

Debian and Fedora's package managers both keep the last three kernels operating and in grub.

(This is a hassle for me since Fedora has had a long string of updates since 2.6.36 which was the last one that gave me working r/w HFS+ support, and i have to keep removing the second-to-last kernel manually with the package manager.)

9mit3t2m9h9a6• 14 years ago

Atomic upgrades. Run an update, interrupt it at random time, reboot. With Nix, there is a very limited set of problems you can get due to that. FS corruption if you do a hard reboot, filesystem mounting problems if you changed your partition layout. But you cannot get a problem with a new libc being partially deployed - you either get new version of everything or old version of everything.

mjs• 14 years ago

I don't understand this. In particular I don't understand the sense in which it is "purely functional". Surely a package manager needs to mutate the state of a system, which seems to be an inherently non-functional operation--? If I have a system S1, and then I do S2 = install(S1, "firefox"), does S1 still exist?

This bit is also interesting: "Runtime dependencies are found by scanning binaries for the hash parts of Nix store paths (such as r8vvq9kq…). This sounds risky, but it works extremely well."

agumonkey3• 14 years ago

I think of it as git-for-system. All the data lies in the store/repository. Nix offers you to pick different isolated branches as needed.

dons• 14 years ago

Installing the "same" package doesn't overwrite previous versions -- all packages are persistent -- so dependencies continue to work.

It is the mutation of packages, silently breaking API and ABI compatibility, that is the bane of distro management. Nix is a great attempt to address this.

ori_b2• 14 years ago

That was solved decades ago with static linking. The bane of distro management is keeping everything as up to date as possible without breaking things.

bumeye2• 14 years ago

Yep S1 still exists. Nix does not do destructive updating. When you install or update a package, you'll always be able to revert to the previous state. The best explanation of that process can be found in the original paper:

Nix: A Safe and Policy-Free System for Software Deployment - http://www.st.ewi.tudelft.nl/~dolstra/pubs/nspfssd-lisa2004-...

I really hope nix will become standard practice, it has some significant advantages over existing package managers.

jerf3• 14 years ago

Purely functional does not mean "has no state at all". This meme needs a stake stuck through it, though beats me how to do it. Functional languages exist and do real work.

In this case it means that packages are handled as functional programming languages handle data values. New ones are created from old ones, and unreferenced values are cleaned up via garbage collection.

thurn• 14 years ago

It doesn't seem unreasonable to think that "purely functional" means "consisting of pure functions". Why not just say "functional" if that's what is meant?

shurane• 14 years ago

The about-nix page is rather helpful on why you would want to use nix:

http://hydra.nixos.org/build/2609700/download/1/manual/#id49...

From my cursory glance over nix's advantages, it sort of looks like the vagrant deployment system, but for package management. Or maybe something akin to Chef and Puppet (of which I have no familiarity with whatsoever).

shasta• 14 years ago

To everyone who "doesn't understand", go read the "about Nix" section of the link.

strictfp• 14 years ago

First yum - "a package manager written in Python" and now Nix - "a purely functional package manager". Well, I for one do not care the least which language or paradigm my package manager is using. What I do care about is compatibility, reliability, speed and correctness. How is Nix improving the situation for the user?

agumonkey3• 14 years ago

Immutability is often helping correctness, and for the user, a simple way to rollback (as mentioned earlier here). I'm trying to use nix as a main system so it's still hypothetical on my side. That said, nix dev team built a continuous integration system for OSes using nix, that can automate system builds on many arch, it's a good sign.

jamii• 14 years ago

"Nix is a purely functional package manager. This means that it treats packages like values in purely functional programming languages such as Haskell — they are built by functions that don’t have side-effects, and they never change after they have been built. "

"You can have multiple versions or variants of a package installed at the same time. This is especially important when different applications have dependencies on different versions of the same package — it prevents the “DLL hell”."

"Nix helps you make sure that package dependency specifications are complete."

"Nix has multi-user support. This means that non-privileged users can securely install software. "

"Since package management operations never overwrite packages in the Nix store but just add new versions in different paths, they are atomic ... And since package aren’t overwritten, the old versions are still there after an upgrade. This means that you can roll back to the old version"

"In addition to downloading binaries automatically if they’re available, Nix can download binary deltas that patch an existing package in the Nix store into a new version."

"Nix can be used not only for rolling out packages, but also complete configurations of services. This is done by treating all the static bits of a service (such as software packages, configuration files, control scripts, static web pages, etc.) as “packages” that can be built by Nix expressions. As a result, all the features above apply to services as well: for instance, you can roll back a web server configuration if a configuration change turns out to be undesirable, you can easily have multiple instances of a service (e.g., a test and production server), and because the whole service is built in a purely functional way from a Nix expression, it is repeatable so you can easily reproduce the service on another machine."

fffggg4• 14 years ago

This sounds nice, and it's a direct quote from the product page, but I can guarantee that Nix does nothing of the sort. This would need to be addressed in ld.so. It's simply not possible for a packaging system to solve this problem on a typical unix system.

chriseidhof2• 14 years ago

Just to clarify: the purely functional part is how the package are managed. Nix itself is mostly written in C++, and I'm not sure, but probably not in a purely functional style.

snprbob86• 14 years ago

I'd really like to see an example of the "Service deployment" approach discussed in the overview.

ef4• 14 years ago

I think that this is a stretch of the term "purely functional".

The relevant property is actually immutability, which looks nice.

While immutability and functional programming often go together, they aren't the same thing.

chriseidhof2• 14 years ago

It is also about the way packages are specified:

(from the About Nix section on top of the page)

bumeye2• 14 years ago

There's also referential transparency.

For programming languages, that means the result of a function is dependent on it's inputs ONLY. The function can not use any unspecified inputs like system time or an internal state.

In Nix, that means a package can only depend on dependencies which are specified beforehand. It's nearly impossible to use libraries or binaries which you did not specify.

That behavior makes it easier to reason about your whole system, like determining which packages are unused. This is similar to the advantages of referential transparency in functional languages

emillon• 14 years ago

> that means the result of a function is dependent on it's inputs ONLY

Not quite, it means that a function call can be replaced by its value (A function can depend only on its inputs, yet write to a global variable, for example).

moonboots3• 14 years ago

The functional approach seems to work great for sandboxing binary artifacts of packages, but configuration files and user data are more difficult to handle. Daniel Burrows, a debian apt developer, wrote an analysis of Nix a while back[1], and he brought up the problematic example of upgrading database configuration file formats. It seems difficult to seamlessly upgrade and downgrade these types of files whose semantics and syntax may change between versions. When these conflicts occur in Debian/Ubuntu, the upgrade process blocks and the user is prompted to resolve the 3-way merge.

I think packakge managers should only handle the storage of static files like executables and shared libraries. Forcing the user or another process to handle configuration files and startup/shutdown hooks has additional benefits such as making these files easier to store in version control.

[1] http://lists.debian.org/debian-devel/2008/12/msg01027.html

zobzu• 14 years ago

That link is actually much more concise, thanks!

edolstra2• 14 years ago

In NixOS (the Linux distribution based on Nix [1]) we use the functional approach for managing all the "static" parts of the system, i.e. packages, but also most configuration files in /etc - these get build by Nix functions in much the same way as packages. This is good since it means that (say) a rollback of the system will cause it to revert to a consistent combination of packages and configuration files. (So you don't get an Apache httpd.conf that refers to some feature that doesn't exist in the Apache you just rolled back to.)

However, when it comes to stateful stuff such as user data or databases, the funtional approach indeed doesn't really apply very well. NixOS manages state in an imperative manner in the same way as every other distribution. (For instance, NixOS uses Upstart jobs to create/update state.) That means that a rollback of your system won't rollback (say) your PostgreSQL database, unless somebody made the Upstart job of PostgreSQL do that. So when it comes to state, NixOS isn't better than other distributions, but it's not worse either ;-)

[1] http://nixos.org/, http://www.st.ewi.tudelft.nl/~dolstra/pubs/nixos-icfp2008-fi...

moonboots3• 14 years ago

Thanks for the response. I agree that some functionality currently handled by package managers is hairy, even for other package managers. To some extent, I don't think this functionality should be managed by package managers. It would hurt usability because the user would need to set up configuration files, hook the package into the init/upstart system, and fix breakages across updates. On the other hand, I've never been a fan of running 'sudo apt-get install apache2' on a development machine and exposing a public port and running an application I don't always need every time I boot up.

nwmcsween• 14 years ago

The problem with the nix package manager that I see is it doesn't integrate into a hierarchy of any kind thus no partitioning of data. Also the nix language is very obtuse and domain specific(which is a good thing as well as bad). I plan on creating a meta package manager that is functional but allows output of arbitrary formats such as rpm, deb, etc. I also think integration is a separate but just as important aspect and hope to use mruby, rspec and augeas(or something else as it seems the parsing format is yet another NIH wheel).

rcthompson2• 14 years ago

Does anyone have experience using Nix (or NixOS) for practical purposes (instead of just testing it out) that they'd like to share?

torkins• 14 years ago

At LogicBlox (http://www.logicblox.com/), we use it to deploy our customers' applications both for testing and production. We also use hydra (http://hydra.nixos.org/) to run our buildfarm. We're soon going to use NixOS for our deployments. If it wasn't clear from that, we're fans... :)

9mit3t2m9h9a6• 14 years ago

I use it on my notebook and it is the only OS I ever booted on it. Works fine for me, but I do like to do strange things with my system. This means that I do value the ability to revert a problematic change and I am OK with configuring things via low-level configuration files.

agumonkey3• 14 years ago

Someone on reddit showed us this Gnome project : OsTree https://live.gnome.org/OSTree/NixOSComparison

darklajid• 14 years ago

So what is the difference to GNU stow [1], if we ignore the 'works a bit like Haskell' description?

I like these kind of snapshots, the ability to install multiple variants. But it's an old hat, no?

1: http://www.gnu.org/software/stow/

rcthompson2• 14 years ago

Nix is a package manager in the same sense as apt and yum. It resolves dependencies, it doesn't just install software.

hristov• 14 years ago

Ok maybe i am missing something, but it seems to me you may want some mutability in a package manager. For example, updates that are simply bug fixes and security fixes but are backward compatible should be substituted for older versions.

Suppose you have a networking library, which is used by several applications. All of a sudden you find a security hole in the networking library. And then you fix it. But the fix does not affect any of the published APIs or functionality and is thus completely backwards compatible for all legal uses.

When you fix the library you release a new version of it. Shouldn't that new version be automatically used for all applications that use the library? If you have an immutable packaging system, all applications will still use the old version of the library and they would have to have their packages explicitly modified to use the new version.

Now this may work for super high value and super secure systems, where someone has the time to individually test the new version of the library with each application that uses it and then individually update each application.

But for the usual desktop system this sounds like a recipe for having a bunch of out-dated hole ridden software, where multiple badly maintained applications still carry known security holes from many years ago; or, in other words, this sounds very much like Windows.

moonboots3• 14 years ago

This is a potential problem, but I think in practice backwards compatible updates and patches for Nix could work very similarly compared to a distro like Debian. For example, when openssl releases a security fix, Debian uses its large, mostly automated test suite [1] to verify that this new update does not break compile and (to a lesser extent) runtime compatibility and correctness.

Instead of an implicit promise of compatibility, a Nix continuous build and testing system[2] could explicitly mark openssl-dependent packages as compatible by bumping their versions. The end user would need to update a lot more packages, but these cost could be reduced by binary diffing old and new packages, which should be almost identical with the exception of dependency metadata.

[1] http://www.youtube.com/watch?v=3aoDjW1bjxo [2] http://nixos.org/hydra/

9mit3t2m9h9a6• 14 years ago

In practice it gives you more options for a price of keeping more junk on HDD. After a security update, all application hashes are calculated using the new default version of a library. So, once you launch an update you get a fresh package set linked against the latest version of the library. For large applications used by many people you are likely to get a small binary patch from the previous version, but this is just an optimization. So the default is just spending a bit more resources on the same behaviour.

What does change, though, is that you can opt to pass another version of the library to a specific application manually. If update from libA v1 to libA v2 breaks AppX, you can easily build AppX-with-libA-v1. It doesn't make any sense for security updates, but for major interface changes it sometimes provides a smoother migration path.

daivd• 14 years ago

Since Nix keeps multiple versions of almost the same binaries, it seems to me that something like ZFS, where several files can share the same physical block on disk, is the perfect match. Is that something you recommend when using NixOS?

Nix 1.0 released: purely functional linux package manager

Related Stories