The future of software is Nix

https://determinate.systems/posts/the-future-is-nix/

by biggestlou • 2 years ago

42 61 2 years ago

grhmc16• 2 years ago

Hey folks CEO of Determinate Systems, and the author of the blog post. I'd be glad to answer questions about the post and what we're doing!

steveklabnik• 2 years ago

I am a Windows user. The default nix implementation does not support Windows.

How can the future of software leave out one of the largest platforms on the planet?

grhmc16• 2 years ago

Hi Steve!

Well. The future isn't evenly distributed yet :). There is work in flight to make Nix support Windows. Also, and I know this doesn't count, but it works great in WSL today.

stackskipton2• 2 years ago

Because the future of software isn't on Windows. Even Microsoft realizes that with MSSQL moving to Linux (and dying), .Net recommended on Linux and internally, vast majority of workloads that can run on Linux are doing so. Apparently, a ton of Office Teams architecture is Linux.

jonringer1172• 2 years ago

(Thanks for all your work on rust)

There is some work to get nix to work with Windows, this is mostly being done by john ericson.

However, the real issue is that nixpkgs (the repository containing all nix expressions) heavily assume unix paradigms.

There could be a "nix-windows repository" in the medium to far future, but it will likely be very divergent from the current nix+nixpkgs story of today.

nalzok• 2 years ago

I agree that Nix is the right idea, but I'm hesitant to adopt it right now because (i) rough edges like poor documentation and inconsistent interfaces make it feel immature, (ii) things are moving very fast with experimental stuff like flakes, which means it will take a while before Nix starts to stabilize, and (iii) recent community forks like Lix might create a fragmented ecosystem in the long run.

What are your thoughts on there concerns?

grhmc16• 2 years ago

I think those are fair perceptions and concerns.

Regarding experimental features like flakes: the reality is they're incredibly stable. I've written about this before: https://determinate.systems/posts/experimental-does-not-mean.... They haven't realistically changed in years, because they work so well. The experimental label is practically FUD at this point.

If the Nix team were to change flakes in a breaking way, it would be stunning neglect for the vast, vast percentage of the ecosystem that has already adopted them. Our data shows that of all the (OSS) repositories created every day, almost 90% of them start with a flake.nix. Of all of those projects, less than 20% use the legacy file formats, and most of those are using the flake-compat library.

On documentation and interfaces, I agree, and we and the greater community are working hard to on that problem. I'll take time, but it is decidedly better than it was a few short years ago.

And on community fragmentation, I just don't see it becoming a problem. The core Nix ecosystem is so large and diverse, I don't see meaningful fragmentation coming out of this.

dpatterbee• 2 years ago

As a response to (ii), I assure you that things are most certainly not moving very fast with experimental stuff like flakes. Flakes were first released as "experimental" almost 3 years ago and have been stuck in feature purgatory ever since.

autoexecbat• 2 years ago

Is there a good book that describes everything we need to know?

grhmc16• 2 years ago

Whew. "Everything" is a tough one, today.

1. I know there is an upcoming book, NixOS in Production. There is a lot of great info in there!

2. Have you seen https://zero-to-nix.com/?

aiono• 2 years ago

https://nix.dev/ is pretty good for getting started these days.

cloudripper• 2 years ago

https://nixos-and-flakes.thiscute.world/ is also an excellent resource.

croes• 2 years ago

Nix is the future gets another meaning if you know that nix in German means nothing.

grhmc16• 2 years ago

Eelco's typical online handle is "niksnut", which obviously means "nix nut" but also "good for nothing" :).

scns• 2 years ago

"Nichtsnutz" in german.

jmull• 2 years ago

It also means nothing in English, though that's somewhat obscure from my perspective. (Looking it up, it sounds like English borrowed it from German a while back.)

In English I think it's most often used as a verb that means basically to cancel. Like, "We're running out of time, we should nix feature X."

andrewla4• 2 years ago

Similarly in English. I think "nix", like "git", is a self-deprecating name. It's a play on both "nix" -- to cancel or to not exist, and "nix" as the suffix for Li- and U-.

smallerfish• 2 years ago

Cool! I'm glad somebody is working on this.

What's your business model?

grhmc16• 2 years ago

A FlakeHub organization with private flakes and cache is $20/user/month, plus at-cost storage and bandwidth. Members of that organization get automatic access to the private flakes and read access to the cache. In the future, we'll be opening up the policy engine to make it more flexible.

uberduper• 2 years ago

I really like Nix package management. I really hate Nix configuration management. I already know how to configure everything. I don't want to have to figure out whatever abstracted config options some random package maintainer decided on.

grhmc16• 2 years ago

I feel that. This is one place where the "nix is everything, everything is nix" mentality hurts the project. NixOS modules are where the abstract config option sauce lives. It isn't an inherent part of Nix.

As Nix (and NixOS) becomes more widely used, this is one place where we'll have to find a way to let users do what they know how, and stop getting in the way.

bsimpson3• 2 years ago

Part of the problem is that the underlying systems aren't Nix, so you're replacing the easy way with the Nix way.

I wouldn't be mad if opening the system preferences and setting my timezone also updated my timezone in Nix. I am mad that I can't change my timezone when I travel, because the GUI picker doesn't know that Nix has taken ownership, so only changes in Nix take effect.

andrewla4• 2 years ago

The future of software is Nix in the same sense that the future of software was Bitkeeper.

It was readily apparent to anyone using Bitkeeper that this was the right way for version control software to work. Similarly, the Nix model is clearly correct.

But just as Bitkeeper was a clumsy early attempt at implementing that paradigm, so is Nix. The article says that "Nix is declarative" which could not be further from the truth. It has a confusing and awkward language with multiple ways of accomplishing the same task, twisting concepts like "function" and "struct" into confusing knots that make it difficult to follow. Debugging is nearly impossible, and even test evaluations take a lot of work to get right. It is riddled with conventions and "helpers" that have idiomatic forms that are not well-documented and frequently abused.

But the model is RIGHT. This is the way to distribute software, this is the way to write software, this is the way to ensure hermetic distributions, this is the way to avoid global implicit state. And Nix as a proof of concept is great because it shows that it does work (basically).

So what is the path forward? Kill the language and replace it with configuration, with external scripts to do the heavy lifting. Make mutations of the data model the fundamental operations rather than having an intentional CLI that conceals the underlying functionality. So make it less usable but more understandable (like the path that Git took) and iterate from there.

grhmc16• 2 years ago

Back in the day, adopting Git was also clumsy and hard to implement. It was buoyed significantly by the investments GitHub made into its usability.

I agree Nix is going to have to evolve to gain wider adoption. It's part of the work we're seeing in the ecosystem, and also work we're already doing :).

adamc• 2 years ago

But git had Linus and Linux on board. What is going to give Nix that credibility?

anotherhue• 2 years ago

I use NixOS in spite of the Nix Language. It's worth it.

grhmc16• 2 years ago

My man.

andrewla4• 2 years ago

Of course -- as do I. But I've become unwilling to accept that the failure of the Nix language is just a skill issue.

jonringer1172• 2 years ago

There's nothing wrong with the fundamentals of Nix.

But working with nixpkgs is quite difficult for most. Which is why I'm attempting to address those issues in a fork, Ekapkgs.

The repos will eventually expose optional flake entry points, so it will be compatible with flakes and non-flakes.

If you're curious, https://www.reddit.com/r/NixOS/comments/1gatci0/announcing_t....

grhmc16• 2 years ago

I love your project, Jon! I think the Nix ecosystem will be improved by having more, smaller things, where the boundaries are clear.

rfoo2• 2 years ago

Kill the language and replace it with configuration

No. Hard no. Please don't tell me you are going to kill a fine (even though awkward) configuration language and replace it with (potentially text-templated) yaml.

Spivak• 2 years ago

I think I would rather replace it with a spec for derivation files implementable in any language that existing build systems can hook into and use as outputs.

    make derivation
    python -m build derivation

Trying to package software with dependencies that don't exist in nixpkgs is painful despite the tooling existing to build and install them.

rossng• 2 years ago

It feels like there might still be time for Nix to morph into something that can get mass adoption. But it probably requires some radical technical changes and the controversy around flakes (which are incredibly useful) suggests that the existing community might not accept that.

Whatever comes next, IMO the killer feature will be that it enables rock solid developer tooling. LSPs that work; typed inputs to modules; control of deployment of NixOS machines and so on.

Ferret7446• 2 years ago

Declarative configuration predates Nix, there are many far more popular options both before and after Nix, and generally declarative configuration is only ever popular/used in a few specific domains/niches.

I remember being enamored by Haskell in college and I asked my professor why pure functional languages aren't more popular. His answer really stuck with me. He said: because reality is stateful.

Reality is stateful. As a consequence, declarative configuration leaks like a sieve, and leaky abstractions suck. When you think about it, declarative configuration is just a DSL for implementing idempotent deployment flow, and idempotent deployment flows in general are quite successful (e.g., Docker, Flatpak, Ansible), and so declarative configuration tends to work okay for restricted subsets of those domains.

tome• 2 years ago

That's a strange objection, because Haskell supports state, of course. In fact I'd say it's the best language for supporting state, because it makes it explicit.

kylewlacy• 2 years ago

There are also quite a few attempts at "Nix without NixLang". In the ecosystem, I can think of Nickel (a different configuration language, but one with strong typing), Garn (Nix but using TypeScript), and Guix (Nix but using Scheme, and also kind of an ecosystem fork I think?). The common link behind all of them is that they all still have Nix in their DNA, i.e. using a Nix store with Nix derivations under the hood

Outside Nix, there's Brioche (my project!), and there's also Vorpal and Tangram, all of which are "Nix-like" but are from-scratch projects instead of using any Nix tooling

So I definitely agree that Nix as an _idea_ is right, and I definitely also agree that NixLang is a blocker for mass adoption. Personally though, I think if you dig deeper, there are other places that could use improvement beyond just the language itself (but nothing that would necessarily be a show-stopper for adoption IMO). But that's why my project is a "Nix-like"-- there are things that I think Nix could / should do, but which aren't really compatible with the way Nix itself works. One example: Brioche works without having a global store path-- it's per-user instead so that it can be installed rootlessly-- which doesn't really map well to how Nix itself handles store paths and dependencies

So point is, I agree Nix is the right model, but I think the _real_ gold lies outside Nix itself (...with the side note that I'm obviously biased!)

andrewla4• 2 years ago

I’ll check out your project - everything you say here resonates with me, especially the rootless nature.

What we probably need, long term, is a content addressable store, local first, secure, and freely shareable. Layering this on top of POSIX filesystems feels awkward and difficult to trust. But this is the fundamental building block of Nix-like systems and would make the move from rootless to system-wide a mostly trivial exercise.

brianjlogan• 2 years ago

That doesn't make sense to me. You can use Nix on non NixOS distros in your user space. Home manager is quite popular.

Have you tried using it in user space?

amykhar• 2 years ago

It's not going to be the future of anything if it doesn't get much better documentation.

grhmc16• 2 years ago

Hear hear. It takes continuous effort not just on documenting, but also reducing the need for documentation by cutting interfaces and accidental complexity.

laerus• 2 years ago

What about immutable systems like Fedora Atomic?

grhmc16• 2 years ago

They're great! But also still miss fundamental pieces that Nix gets right. Especially the way the build and dependency graph goes right to the root of every package, and comes together to completely describe the system you're running today.

abdullahkhalids• 2 years ago

To make a smooth path from “Nix could solve my dev, build, and configuration management problems” to “Nix is solving these problems” and in a way that doesn’t involve turning someone from a developer to “the Nix person.”

When do we get Nix easy enough for not just software engineers, but also scientists etc who need reliable reproducible systems for running simulations with zero hassle?

grhmc16• 2 years ago

Hell yeah. Speaking my language! I don't know, but we need that!

Koshkin• 2 years ago

The future is not what it used to be.

johnklos• 2 years ago

...and it never was.

gr4vityWall• 2 years ago

FWIW, the post looks more like a startup pitch from its founder than a more technical article.

My take as of now is that I like what Nix achieves, but I don't feel the complexity is worth it. Docker and similar container tech have issues, yes, but I'd have way way less trouble onboarding a dev on it than Nix.

I hope that changes in the future, though.

grhmc16• 2 years ago

I mean, it sort of is, on the heals of two other announcements earlier this week:

* https://determinate.systems/posts/announcing-determinate-nix...

* https://determinate.systems/posts/flakehub-cache-and-private...

rgreekguy• 2 years ago

Something I have never understood is why the hard-on with Nix, and Guix is not heard of even a tenth of the times. Is GPL really so repulsive to the masses? Or the libre kernel default?

sleepydog• 2 years ago

I think guix was just late to the party and the extra steps to setup the nonguix for the non free software like steam is enough of a deterrent for users to go to nix instead.

I use guix myself and prefer its use of scheme. I don't mind having a little less guidance and "how-to" articles available if I'm trying something new because I'm comfortable reading its codebase.

eikenberry• 2 years ago

How is Guix's repository? One of Nix's biggest draws (I'd say the biggest) is it's expansive package repository.

dqv• 2 years ago

That's what it was for me. I'm really fuzzy on the details because this was years ago. When I first heard about Guix, I wanted to try it (even though I didn't understand the benefits) but there was some unfree part I needed to install it. And ideologically they were against that. There either weren't good tutorials on how to bypass it or it was not even possible. So I just didn't install it.

FWIW Nix does not allow unfree packages by default or, at least, that's what having to add a package to allowUnfreePredicate makes it feel like.

stackskipton2• 2 years ago

We have Nixstan on "Ops Team" who keeps pushing us to use it. We haven't seen need that much.

So can someone give me elevator pitch that if we are CRUD Web App, why Nix instead of Puppet/Ansible + Docker Containers?

Also, this from the blog article: "What if adopting Nix didn’t irritate your IT or security teams?" Too late, security team is already pissed. They did some scan, we pinned a package, it has 3 high severity and it needs to be updated tomorrow. So yea, we are just yoloing package updates, we don't have much of a choice.

Oh yea, we have some third party proprietary package we use. Support told us, Red Hat/Rocky/Ubuntu or we could kiss support goodbye. Any suggestions that doesn't involve shoehorning nix into Ubuntu which is like mud wrestling with a pig?

thraxil• 2 years ago

So can someone give me elevator pitch that if we are CRUD Web App, why Nix instead of Puppet/Ansible + Docker Containers?

For better or worse, Nix forces you to work out the exact dependency tree of your entire system. It can be a pain but once you have it, if one part of that tree changes, nix can rebuild only and exactly the parts of the system that depend on it. That goes all the way through from packages and libraries to systemd services and application config files. That happens safely, reliably, is extremely cacheable, and is done in a way that can actually be perfectly reverted to a previous version. You can build docker images with nix and the result is very small minimal images (it's effectively a `FROM scratch` every time but with a potentially even better layer structure that again can be more cacheable than a typical Dockerfile setup).

From a security perspective, Nix can be a huge win since it gives you such precise control over what's installed on your system and makes it simple to track, every part of it checks cryptographic hashes, etc. But yeah, most security teams don't know what to do with it yet and probably don't have the tools to deal with it yet.

bsimpson3• 2 years ago

Our world’s infrastructure should not be stuck in a “don’t touch it, it works” mode, but it is.

The future of software is Nix

Related Stories