ronef

We've been working quite a bit on bringing in Nix/Nixpkg/Nixos based architecture into the space of bringing in determinism to the baseline infrastructure of our AI stacks. This also leans into the conversation around sovereign AI and how Nix is letting us own more of that stack.

Asking this with a bias since I work on Nixos.org and Flox.dev - How is the team thinking about the infra layers underneath these models? Any priority or reason to imbed determinism/reproducibility at the bottom of the stack?

What's the best practice right now for setting these up? We've been primarily using Nix/Flox to set up the models pretty quickly and at least with minimized amount of commands(biased Nix/Floxer) here and found it useful

A small note of amazement from Nix/Flox person here. Incredible to see this release and congrats! Mike, you're an allstar for so many years of contributions!

I.e. is this overhyped?

I feel we should definitely be digging way beyond the SBOM... but also wondering if the forecasting in the general ecosystem is on point or not.

Ron from Flox here. We recently wrote this piece on how Nix can address the emerging CVE remediation problem. We are curious to hear any feedback on our approach.

[Disclaimer: biased Flox/Nix person]

There's a lot of reasons to use Nix instead of or WITH Homebrew depending on your exact needs.

Where it’s paid off for me (and where I think it actually wins) is when the problem is recreating environments: multiple machines, teammates, CI, nasty native deps, CUDA stacks, etc. At that point you’re choosing where entropy lives: in invisible drift (brew/manual installs) or in a repo you can diff/rollback.

Also, you don’t always need to go full “immutable everything.” Really depends on your needs here. Hybrid tends to be another sane path. In certain situations this can get you 80% of the upside without having to rip it all out. So kinda the "good enough" which I've seen a lot of folks do.

We (Flox) actually worked on this with Kelsey Hightower a while back - https://bsky.app/profile/kelseyhightower.com/post/3ld2rsccls...

Highly recommend to check this out, the blog/Arnoult does an amazing job in very succinctly breaking down the aspects of SBOMs in a Nix based infra approach. We can go way beyond the current SLSA levels and provide full provenance at the atomic level of the supply chain for when it's needed. And as Arnoult points out, prune when it's not. There's good work being done on this across the Nix ecosystem and we have also seen a lot of use for it come in through Flox as well!

And I'm back in the land of the living. Can't really beat a response from Justin Cormack!

Jotting down a few quick thoughts here but we can totally go deep. This is something Michael Brantley started working on a few months ago to test out how to make it super easy to ease and leverage existing Nix & Flox architecture. One of the core differences from my quick perspective is that it specifically leverages the unique way that Flox environments are rendered without performing a nix evaluation, making it safe and optimally performant for the k8s node to realize the packages directly on the node, outside of a container.

online community love was not in my cards going into day 3 of a newborn but I'll take it + definitely needed! thank you!

Yes, this hits the nail on the head. We’ve seen the same explosion in image size and rebuild complexity, especially with AI/ML workloads where Python + CUDA + random pip wheels + system libs = image bloat and massive rebuilds.

With the Kubernetes shim, you can run the hash-pinned environments without building or pulling an image at all. It starts the pod with a stub, then activates the exact runtime from a node-local store.

Going to sound weird but with both my hats on I super appreciate this perspective. I can only speak to some areas of Nix and Flox obviously and I know folks are looking into doing this to your point a whole lot better. Zooming in way more into solving for us that just want to run and fix it fast when it breaks.

Also, think it's a huge ecosystem win for FreeBSD pushing on reproducibility too. I think we are trending in a direction where this just becomes a critical principle for certain stacks. (also needed when you dive into AI stacks/infra...)

Ron from Flox here, woke up to feed a brand new 3 day old to see this here! On about 3 hours of sleep (over the lat 48 hours) but excited to try and answer some questions! Feel free to also drop any below <3

We did just launch this last week after a good bit of work from the team. Steve wrote up a deeper technical dive here if anyone is interested - https://flox.dev/blog/kubernetes-uncontained-explained-unloc...

+1 to Farid, great write-up! What you’re seeing is the long-standing “deriver” mismatch: fixed-output derivations can change their .drv without changing the output path. Eelco is calling it out as well in the comment below. I believe the idea behind the path forward is there but happy to hear more!

Also. Check out Farid's other posts.

Bellroy is. very cool and deep in tech. We actually did a Nix in the Wild with them a while back! https://flox.dev/nixinthewild/nix-in-the-wild-bellroy/

I've run into similar pain points with GitHub Actions in past roles but I still very much use them/get value. One approach that's helped us at Flox is indeed using Nix and we've now seen customers start leveraging that. Significant drop in “works on my machine” issues and more reliable CI pipelines, etc... It’s not a silver bullet, but it offers a solid technical foundation for tackling these challenges. On the Flox side we are very much on the integrate and improve rather than full replace for scenarios like this one <3 Happy to answer any Nix items on this!

To add on to this, more than just useful, incredible team building it!

Awesome to see this on here! We've seen a lot of comments talk about Nix versus other tools recently and wanted to share some of our thoughts on it. We like seeing Nix meet developers where they are, integrating into our existing workflows, including containerized ones.

Hey from Berlin! I'm Ron one of the founders at Flox and also board member of the NixOS Foundation. Just noticed folks are talking here and that's super exciting! We are all in the midst of the yearly NixCon so apologies if I'm slow to respond or my answers are a bit shorter. Definitely happy to dig deeper!

I saw your question and actually wanted to share the response I shared a few months ago, happy to unpack or talk about any of it as well.

This is from our original 1.0 announcement here (https://news.ycombinator.com/item?id=39692801) -> bringing forward what you saw today for free and open source, was a major part of why I started Flox, with much more to come into it. What we released today will be free forever (both the open source client and the FloxHub services for sharing environments). We plan on expanding the offering to include a more robust private software catalogs that layers on top of the OOTB Flox Catalog that ships with Flox. If you are interested in publishing your output or need revised versions of open source packages in Flox then it'll be very easy to have your own catalog to compliment the always-free Flox Catalog that Flox ships with today. Beyond that, we are focused on a number of services that help bring Nix's build to the enterprise. Over time we intend to sell a solution to enterprises - through subscriptions and services - so they can more effectively manage expansive and fragmented software supply chains. As part of developing custom tooling for enterprises, we think it's reasonable for them to participate in funding for that work.

There's a bunch of tutorials out there now, but what's the main case you're trying to solve for? Just an easy dev environment? There's an ecosystem out there that ease up the entry for Nix and also nix.dev does a great job as part of documentation efforts inside of the community.

As a Nix foundation person - If anyone gets a 100 second pitch for Nix, dinners on me and I'm using it in the Nix marketing efforts :D

Agreed on the packaging experience for nix. We've been talking about that for a while also on the Flox side when we're working with folks.

I love this. I believe I might have even interfaced with your team around that time. I was leading Facebook's (now Meta) Developer Products team and we were building against super similar areas internally.

We ran back then a similar project that I coined "Developer On-Demand" to tackle that same problem space. It's also what eventually lead me to find the magics of Nix and then build Flox.

I also agree with a lot of what was shared in other comments, while the problems we tackled at large orgs such as Facebook, Shopify, Uber, Google (to name a few teams I remember working with) and obviously also Stripe, certain areas of the pain are 100% universal regardless of team size.

On the Flox side, we're trying to help with a few of them today and many more hopefully in the soon future, very open for thoughts! Things like - simple to use Nix for each of your projects + keep deps and config up to date across everyones Macbooks and Linux boxes, etc -- even if you don't have a full AWS team and Language Server team ready to support.

While I personally don't love developing at all times within a container. When I need to, I've found the flow of setting up with Nix and then containerizing to be the most ideal today. Is there something new recently that folks are using?

(I'm also a biased person as I'm a Nix person at heart)

We were building something very similar to this in collaboration with the VSCode teams a while back at Meta. The goal was to get development on demand via hooking into some beefy linux server farm shile having VSCode stay as the local experience. Though one of the problems even back then was similar to what you are mentioning. It did however remove all last mile network reliance during lockdown which was a plus.

Ron from flox.dev here, the note brought a lot of smiles across the team. We've been working on this for a while now and would love to hear if there is anything we can prioritize or do to keep making it better.

Thanks for asking! This is definitely a deeper lets get beers convo when you're in the bay! But - Have you ever seen the Microservices video? https://www.youtube.com/watch?v=y8OnoxKotPQ

I was leading the developer products team at Facebook back then and started a project around injecting remote capabilities into local dev. Short of it was that thousands of developers were waiting 45 minutes for cold builds... Part of the initial steps for the project was to map out the entire SDLC (so we can get a sense of what parts of the toolchains we need to rebuild), that's where that video above comes into mind if you look at that whiteboard towards the end... Visualizing how complex we've made it is what got me into the headspace of - this can't actually be how we go about this.

Appreciate the call out. Regarding pricing, bringing forward what you saw today for free and open source, was a major part of why I started Flox, with much more to come into it. What we released today will be free forever (both the open source client and the FloxHub services for sharing environments). We plan on expanding the offering to include a more robust private software catalogs that layers on top of the OOTB Flox Catalog that ships with Flox. If you are interested in publishing your output or need revised versions of open source packages in Flox then it'll be very easy to have your own catalog to compliment the always-free Flox Catalog that Flox ships with today. Beyond that, we are focused on a number of services that help bring Nix's build to the enterprise. Over time we intend to sell a solution to enterprises - through subscriptions and services - so they can more effectively manage expansive and fragmented software supply chains. As part of developing custom tooling for enterprises, we think it's reasonable for them to participate in funding for that work.

we will always allow the CLI to stand alone for a local developer or CI runner. the service side of Flox is for those who want a faster performance, easy sharing, or eventually enterprise control. so while the CLI can use the closed service it's purely optional (and will always be optional)