Using NetBSD’s pkgsrc everywhere I can

https://rubenerd.com/using-netbsds-pkgsrc-everywhere-i-can/

by jayp1418 • 5 years ago

104 79 5 years ago

inamiyar• 5 years ago

Very interesting, I didn't know pkgsrc could be used on Linux, I'll check it out. In general the only things keeping me from BSD are non free apps like Zoom so the next best thing is to BSDify my systems.

xaduha3• 5 years ago

I'd try nixpkgs before trying pkgsrc if for some reason I wasn't satisfied with ones that come with the distro. Not sure why BSDfying packages is desirable by itself.

1vuio0pswjnm72• 5 years ago

Nothing against nixpkgs, but if the user is curious about how things are done in BSD perhaps he should try xbps before nixpkgs. It was written by a former NetBSD committer who wrote some useful programs for NetBSD's userland. I am biased perhaps but I believe it is faster than any other Linux package manager. Always looking for something faster so I am welcome to being proven wrong on that.

https://github.com/void-linux/xbps

washbear6• 5 years ago

One reason to try it might be that pkgsrc mostly solves the problems Nix solves with conflicts in a much simpler way, by allowing multiple co-existing branches installed to different prefixes.

johnisgood3• 5 years ago

pkgsrc does work on Linux, but pkgin does not. I have no idea how to update my mirror and so forth using pkgsrc alone. Do not I need pkgin? I cannot do `pkg_add` at all.

I tried to make pkgin work, too, and I think it is possible to make it work with libbsd but I do not have the time. I wonder if anyone has done it.

jperkin2• 5 years ago

I am the pkgin maintainer, and would be interested to hear about any issues you have building pkgin. It should build on any of the 23 platforms that pkgsrc supports, including Linux.

I even offer daily binary package builds for CentOS here: https://pkgsrc.joyent.com/install-on-linux/

Feel free to open up an issue at https://github.com/NetBSDfr/pkgin/issues and we can take a look.

LAC-Tech• 5 years ago

I've been thinking of delegating mandatory crapware like zoom to my mobile phone. Needing to run stuff like that a few times a year is probably not worth preventing me from using an OS.

xaduha3• 5 years ago

No mention of DragonflyBSD? They used pkgsrc as their main package manager for 10 years, portable or not, it still required quite a bit of maintenance. Seems like it was too much in the end, second-class citizen rights are not great.

cpach• 5 years ago

Did they migrate away from pkgsrc? If so, do you know any specifics about why?

xaduha3• 5 years ago

https://lists.dragonflybsd.org/pipermail/users/2013-Septembe...

pxc13• 5 years ago

I'd been wondering how good pgksrc is on macOS for a little while, previously having used Homebrew wherever Nixpkgs was inconvenient or incomplete. Could pkgsrc could be a better choice?

pkgsrc is relatively nice for a cross-platform, source-based package manager in the old style, in that it provides convenient binary caches for most use cases.

I wonder: Has the writer of the post in the OP tried any next-gen source-based package managers, where the kind of isolation he describes as desirable between system and user packages is guaranteed between all packages altogether?

nerdponx• 5 years ago

I'm always interested to try different things, but I don't understand why people seem to dislike Homebrew so much. I've been a mostly-happy user on both Mac and Linux for years.

kelp2• 5 years ago

I dislike it because it’s really slow to do everything. Even printing help text is slow.

Contrast with pacman on Arch, which is incredibly fast at nearly every operation.

Unfortunately I’ve found all the package managers for macOS to be lacking.

brew is slow, but has packages for most things and generally up to date.

fink packages seem a bit out dated last I checked. At least the ones I cared about.

MacPorts is also slow and somewhat out of date.

pkgsrc doesn’t have Apple M1 binary packages available yet.

I haven’t tried Nix.

Sometimes I’m tempted to try my hand at writing my own package manager for macOS, but getting a critical mass of packaged software seems quite hard.

mixmastamyk3• 5 years ago

Built in telemetry/analytics was enough for me to not touch it with a ten foot pole.

mulmen3• 5 years ago

In addition to the valid criticisms in sibling comments:

I don’t like that it changes the rules on file ownership in /opt and /usr/local.

And it’s petty but I really don’t appreciate the beer brewing metaphor. It doesn’t fit and it feels forced. Giving the project a searchable name is fine but cutesy names are irritating when I have real work to do. Just name the tool and especially commands for what they do.

I say this as a person who loves beer.

pxc13• 5 years ago

It's got a larger user and contributor base on macOS than competing package managers, and that works in its favor, especially in terms of package selection. I'm not a total hater; there are people I respect who prefer Homebrew over tools that I like better for reasons I understand. And package management is a peculiar interest of mine, so I care a lot about details and there isn't any package management system that totally satisfies me.

That said, every time I've used it, I've been struck by design flaws that make Homebrew feel incomplete or immature compared to most Linux system package managers. Some examples:

• The last time I used it, Homebrew didn't even support recursive uninstallation of orphaned dependencies, guaranteeing the accrual of entire packages of cruft over time. This was apparently finally fixed (in the crummy way that apt-get, the worst of all the Linux system package management tools, handles it) only in 2020 [1].

• A secondary package manager taking over all of /usr/local is quite presumptous and not portable at all

• Homebrew's combination of an assumption of a single-user setup with the choice of installing to global directories in the FHS is awful, and imo it's straight up deceptive to claim that Homebrew's default setup meaningfully supports ‘unprivileged package installation’

• `brew cask` is useful, but the cleavage between it and the rest of Homebrew is awkward, and it's more like an installer-fetcher à la Chocolatey than a real package management system

• Homebrew's packages are built with a relatively low degree of isolation from the base system's libraries and toolchain, so it's not unusual for a macOS upgrade to break installed Homebrew packages, and the build outputs and bottles associated with a package are given per-macOS release. (To be fair, the former is a pretty common problem for all kinds of software on macOS, as well.)

• I really don't like Linuxbrew's abuse of /home, which you have to stick with if you want to use bottles (binary caches)

• Homebrew just doesn't support managing dependencies on libraries in some languages (e.g., Perl) at all [2]— even though it includes packages that depend on them— which fails to meet my bare minimum expectations for a general-purpose package manager. In practice, this means that you have to install additional, language-specific package managers to use Homebrew; you have to manage dependencies in them yourself; and Homebrew packages require you to install these dependencies in system-wide locations, polluting your whole operating system (a potential source of dependency hell, as well as a practical nuisance) for their sake. The latter also, of course, undercuts the unprivileged installation claim that Homebrew touts.

To be fair to Homebrew, it's been a while since I used a Mac on a daily basis, and I expect that Homebrew has improved in some ways in the meantime. I know for certain that it's improved, to reiterate an example given earlier, with respect to some, if not all (i.e., not those managed externally) orphaned dependencies. And Homebrew also has some strengths, namely its very slick and orderly command-line interface.

But the real reasons I try to use Homebrew as little as possible aren't about Homebrew being _bad_ so much as my preference for tools with other strengths (which require a design different from Homebrew's to implement). On macOS, I try to use Nixpkgs and nix-darwin as much as possible because I prefer things like:

• declarative configuration rather than imperative state management

• fully isolated builds, more predictable solutions

• using a single package management tool for libraries and apps of all languages

• building .apps from source like any other package

• atomicity of upgrades

• freedom to roll back configurations/installations at any time

• ability to mix and match packages from different releases, modify dependencies for one project without affecting another

• never installing language interpreters or libraries globally, except for login shells

I don't really want to use anything that doesn't have those features, and for that on macOS, Nix is the only game in town. Homebrew is a pile of state to manage, and I'd rather have intentions to declare than a pile of state to manage. That issue isn't unique to Homebrew, of course.

Pkgsrc doesn't share some of Homebrew's (in my view) other defects, like its abuse of the FHS on Linux, single-user design/bad permissions configuration, building many packages directly on top of macOS system libraries that change from OS release to OS release when it doesn't have to, or the expectation that library dependencies for some languages will be managed manually/externally (and in global directories, to top it off), so that's what makes Pkgsrc interesting to me as an alternative to Homebrew, even though Nix will still always be my first-line choice on macOS.

—

1: https://github.com/Homebrew/brew/commit/0fa417706a2143dbec1e...

2: https://docs.brew.sh/Gems,-Eggs-and-Perl-Modules

1vuio0pswjnm72• 5 years ago

What are the dependencies of these "next-gen" package managers. pkgsrc itself only needs C and a POSIX-like shell.

pxc13• 5 years ago

Fair question!

The two that I know of currently are Nix and Guix. Both target fewer platforms than Pkgsrc, so we know right out of the gate that they're not suitable for all Pkgsrc use cases.

The question of what they depend on needs fleshing out before it can be answered. Both Nix and Guix package software in a way that is, in general, hermetic. This means that they don't assume the installation of anything outside their domain— they don't require you to have anything installed on the base system to get going with them. They don't use your system's shell and they don't use your system's C library. As an exception on macOS, Nixpkgs allows some impurities for some macOS packages, e.g., they are allowed to call xcodebuild. But in general, they include their own toolchains in their entireties. So if what you're getting at is ‘where do they depend on the base system during operation’, the answer is, more or less ‘nowhere’.

If the real thrust of your question is rather ‘what do you need to build them from source’, the answer, for now, in both cases is ‘a bunch of shit that they kindly prepackage for you in a binary tarball, if you want to bootstrap your own Nix or Guix on a given platform’. Guix has been doing a ton of work to reduce the size of their bootstrap seed, and it seems like within a few years they'll have it down to a hex monitor from which they can build a series of Scheme interpreters and C compilers until they have the whole system.

If the main thing you're asking about is ‘what do they use internally to process their package sets’ or something like ‘what's the minimum installed size of these package managers’: for Nix/Nixpkgs you need the Nix language interpreter (written in C++), plus at least what's packaged in the ‘standard environment’ for Nixpkgs (GNU bash and coreutils, plus a libc and a C compiler which vary by platform); and for Guix you need Guile, an implementation (interpreter and runtime) of Scheme. I've never worried about their installed sizes, since they're much smaller than the total size of everything you might want to install with them.

Idk the size of the full dependency closure for Guix and I don't have a great way to look it up at the moment. In the Nix case, the binary installation tarball is ~17 MB compressed and ~72 MB uncompressed, when you install it.[1]

Happily, if Pkgsrc is easier to get started with due to its smaller footprint, it looks like in the future users will be able to use Pkgsrc to install Nix.[2] :)

(For my part, I wouldn't mind if Nixpkgs included Pkgsrc, too. I wish it included Guix.)

—

1: https://releases.nixos.org/?prefix=nix/nix-2.3.9/

2: https://pkgsrc.se/wip/nix

jayp1418• 5 years ago

Check this : https://pkgsrc.joyent.com/install-on-osx/

johnklos4• 5 years ago

Other package management systems do one or two of the following things, but I haven't found any that can do all three at once, other than pkgsrc:

1) have good availability of pre-built binary packages for most platforms

2) facilitate building of packages directly from source, whether on non-mainstream platforms / architectures, and / or with or for the purpose of non-default package options

3) unprivileged builds / installations, packages rooted anywhere in filesystem, completely self-contained dependencies

Having to use more than one package management system because one or the other doesn't do one of these things can be a real pain, and can make maintenance rather difficult.

fiddlerwoaroof2• 5 years ago

nix does all three of these, I think.

fiddlerwoaroof2• 5 years ago

I guess 3 is sort of anti-nix, but I think nix effectively has this with the way nix-build/nix-shell work.

johnklos4• 5 years ago

It looks interesting. However, documentation is sparse. It doesn't clearly say what platforms are supported other than macOS and NixOS (which implies Linux in general, but doesn't clearly state it). Also, it seems to only target x86 and amd64. No mention is made of any other architecture, nor of any BSD other that macOS.

Do you have more information?

johnklos4• 5 years ago

How does one bootstrap nixpkgs from source? Trying to install nix gives:

sh install-nix-2.3.10 install-nix-2.3.10: sorry, there is no binary distribution of Nix for your platform

washbear6• 5 years ago

It is nowhere near as cross-platform as pkgsrc - it only really supports macOS and Linux.

choeger• 5 years ago

Unprivileged works now?

rakoo• 5 years ago

I think Archlinux's PKGBUILDs can do that (https://wiki.archlinux.org/title/Creating_packages#Creating_...):

- With both the community-maintained packages and the AUR packages, you can already find almost any binary that already exists

- If it doesn't exist, or if you need particular options/architectures, a PKGBUILD is basically a glorified Makefile with standard steps, so you can for example get binaries from a http source and use them directly, or pass exotic compilation flags during the build() phase

- It is of course possible to put dependencies inside a given package

- preparing a package is entirely done by your user, no need for root right. Inside the package you put your files in a fake root, and when the package is done you apply the package to your real fs. By default it is installed under /, so of course you need root rights, but you can install it under any root you desire. This is actually what's done when installing Arch: it basically runs pacman (the package manager) but instead of installing packages in the live environment's root, it installs packages in the mounted filesystem that will become your root after rebooting.

Foxboron• 5 years ago

We are missing unprivileged builds and clean chroot builds for `makepkg`. Our interal devtools uses `systemd-nspawn` which requires root. But that should be easy to implement with the leftover code we tried to use with our reproducible builds tooling.

https://github.com/archlinux/archlinux-repro/pull/70

Maybe a weekend hack to wrap `makepkg` with this.

sally1620• 5 years ago

I wonder if anyone is going to create a linux distribution with a base system baked in with pkgsrc managing the applications. It would be very useful for containers and WSL.

znpy2• 5 years ago

There used to be one, which was based on Slackware (of course). If i find the name of that distro i'll either edit this post or reply to myself :)

I tried hand-rolling something similar with slackware 11 or 12 (can't remember now).

jmclnx• 5 years ago

I always liked its concept and I am surprised very few projects are using it. On NetBSD it works well, I guess NIH (https://en.wikipedia.org/wiki/Not_invented_here) syndrome :(

washbear6• 5 years ago

Such a thing exists - https://github.com/oasislinux/oasis

kchoudhu• 5 years ago

I've been using pkgsrc to package our firm's stuff for Mac OS X clients, it works a charm.

https://www.anserinae.net/setting-up-a-pkgsrc-repository.htm...

Currently working on using it to distribute the same software to Linux. Will be great when it gets there, but we aren't quite ready yet.

ehw3• 5 years ago

I really like pkgsrc. I'm not really qualified to give a technical justification for it. It just had the right simple, clean feel to it, which NetBSD in general has. I would even go so far as to say that it was fun to use.

I consequently once spent quite some time trying to get it working under Cygwin so I could use it at work, but never got past problems with Cygwin group names that have spaces in them. It was too complex for me to fix....

protomyth• 5 years ago

Does pkgsrc have "port variants" like macports?

zephyr9• 5 years ago

Not for prebuilt binaries (variants are just separate packages) but because the pkgsrc repo is just a collection of Makefiles, you can cd into the directory and run

    bmake show-options

Then set PKG_OPTIONS.somepkg and

    bmake install

Or even just edit the Makefile.

guessbest2• 5 years ago

I wish there was a package manager for kde and konqueror for the mac

pxc13• 5 years ago

There's a Homebrew tap for KDE! I'm not sure what the present state of it is, but it's here if you wanna try it out:

https://invent.kde.org/packaging/homebrew-kde

guessbest2• 5 years ago

Thanks. I'll look in to this.

weitzj• 5 years ago

I used it with Netbsd 1 and it was the first thing to install in Solaris.

Other question: how does one handle detection whether upgrades exist? Something like renovatebot but using pkgsrc. Or what is the best way in 2021 to trigger fresh builds against the latest packages?

plextoria• 5 years ago

pkg_rolling-replace -u

tingletech• 5 years ago

it was a big help in Solaris to VWware migrations I did circa 2008-2012. I had one bash script that would install the whole site on either, bootstrapping almost everything from pkgsrc.

klibertp2• 5 years ago

From pkgsrc homepage:

You can checkout pkgsrc HEAD with CVS:

...really? Not even Subversion, but CVS?

jperkin2• 5 years ago

The repository of record is CVS, but you can - like many of us do - use either the hg export (https://anonhg.netbsd.org/pkgsrc) or GitHub (https://github.com/netbsd/pkgsrc) instead.

At some point we will transition the repository of record to something else, but it's being done carefully. We want to do it right, and there aren't many volunteers willing help with the work.

klibertp2• 5 years ago

No, I was just surprised - I don't know anything about NetBSD (nor pkgsrc for that matter), but I worked with FreeBSD some 15 years ago and there were plans to move to SVN even back then. A quick search told me that the switch was completed in 2008 (a little bit after I left, begrudgingly, for Linux).

What challenges are there in switching to another VCS? Is the difficulty coming from the need to preserve history, or the need to re-evaluate workflows around the repo, or something else entirely?

Using NetBSD’s pkgsrc everywhere I can

Related Stories