Gittup, a Linux distro on top of Git

by traverseda • 9 years ago

171 67 9 years ago

traverseda5• 9 years ago

I thought this was a pretty useful project for embedded work, and despite it being a few years old I'd never heard of it.

The documentation is a bit... eclectic.

Tup is a build system that keeps track of what files a command accesses. So if you run make from tup, it can look at every file your makefile reads from, and re-run make if any of those files change. Not that useful, in that example, but when you have it run gcc to make each source file instead, it can automatically rebuild just the files that need to be rebuilt.

Building on top of that, they've set up a git repo with the appropriate "tupfiles" (makefiles) to build linux and enough userspace to get by. It's not so much a linux distribution as a build-system that can build an entire linux distro from source.

The big difference between this and a normal build system is that all your source files can be in the same tree as git submodules, and `tup upd` will rebuild the entire thing, only recompiling code who's source file or shared libraries have changed.

I think it could be a pretty powerful system for a lot of projects, I'm particularly interested in using it for building android images, and firmware for ereaders.

xiaq• 9 years ago

It's not so much a linux distribution as a build-system that can build an entire linux distro from source.

Isn't a Linux distro largely defined by its packaging system, which in turn is largely defined by how software is built from source?

traverseda5• 9 years ago

The big difference being that in order to get any real benefit you can't use upstream's build system, you need to use tup. I'd say it's not a linux distro yet.

kbenson2• 9 years ago

I wouldn't say a Linux distro package system is largely defined by how software is built form source, but it can be, and it at least matters in all cases.

I would say the package dependency tree is probably the most important, defining feature of the package system. Second to that is delivery (and the fact it's so high on the list is a testament to its importance, as I can remember a time before it when we just had RPM and dpkg). Some package managers also build form source, but that's a quirk of hose those operate, not really a core requirement to delivering binaries to the end user.

Even FreeBSD's ports build system is largely defined be its dependency tree and the rules they have in place to make fetching and building easy. Whether binaries are delivered from it or source that's compiled locally is somewhat transparent to the end user, other than the extra time it takes.

equalunique• 9 years ago

Page says it's only for x86, but hopefully this will be extended to other architectures. :D

ericfrederich• 9 years ago

Seems like it might be useful in creating a Docker image from source.

nemo1618• 9 years ago

The example of modifying `ls` struck me as something that ought to be possible in any Linux distribution with minimal effort. Instead, we have a situation where most people use binary packages, so the source isn't even available to modify. And even if you run a source-based distro like Gentoo, editing those sources by hand can still be quite intimidating. Version control makes it easier to experiment fearlessly (although it's still possible to brick your system...).

seiferteric• 9 years ago

It's really not too hard on a Debian system to build a modified package. All sources should be available if you have the src repos in your sources.list. You just download the src package, modify, rebuild and install. https://wiki.debian.org/BuildingTutorial

traverseda5• 9 years ago

Return 403 for me...

traverseda5• 9 years ago

It could go a long way towards getting people involved in FOSS systems development.

I think you could avoid a lot of those bricking issues using btrfs snapshots.

deckar01• 9 years ago

btrfs snapshots are the source of all of my OpenSUSE crashes. I'm not sure what the issue is, but eventually it fills up my disk silently with snapshots until the system grinds to a halt.

andrey_utkin2• 9 years ago

And even if you run a source-based distro like Gentoo, editing those sources by hand can still be quite intimidating.

Not so much hassle.

Way 1:

1. put your patch to /etc/portage/patches/sys-apps/coreutils/my-ls.patch

2. emerge coreutils

Way 2:

1. ebuild /usr/portage/sys-apps/coreutils/coreutils-8.28-r1.ebuild prepare

2. edit sources in /var/tmp/portage/sys-apps/coreutils

3. ebuild /usr/portage/sys-apps/coreutils/coreutils-8.28-r1.ebuild merge

Really nice to work this way if you debug compiled software a lot.

unixhero• 9 years ago

Cool.

But why Linux kernel v2.6?

aidenn05• 9 years ago

Because that was current the last time the kernel package was updated?

traverseda5• 9 years ago

It looks like it's an ancient half-abandoned project. Hopefully some people here take an interest and breath a bit more life into it.

ComputerGuru4• 9 years ago

The entire source tree is based off of git submodules and an automatic build system, right? I wonder what would happen if you ran “git submodule —update” (presuming the submodules point to master, though I am unsure if that functionality was around in git when this was published) and then tried to build it with tup.

jwilk4• 9 years ago

The article appears to be from 2012:

  $ curl -Is http://gittup.org/gittup/ | grep ^Last-Modified:
  Last-Modified: Fri, 23 Mar 2012 23:17:45 GMT

dahart• 9 years ago

I'll admit it started skeptical based on the headline, but reading the page, I changed my mind, this seems like something worth at least trying to see what package management feels like. It'd be pretty cool if something like this could help smooth out package version incompatibilities.

In particular this idea is intriguing (and the honesty about not having tried it is appreciated):

Bisection!
Let's say your compy86 is running fine, and then you update to get the latest, and something breaks. You can use git bisect across your entire linux distribution to find the commit that broke it.
Note: It may not work across config changes, since tup doesn't automatically run 'oldconfig' when necessary.
Note: I've never actually tried this, so we'll just assume it works.

orblivion• 9 years ago

Seems like a great way to help ease contribution. I've never used NixOS, but I wonder how it would compare in this and other ways.

aidenn05• 9 years ago

I've been using NixOS for a while now. There is a lot to be said for using nix as a development tool &c, but in terms of a daily driver OS:

Upside: It really is the best of both worlds for binary/source distros. 99% of the time it just downloads an archive and unpacks it, so it's just as fast to install or upgrade as e.g. arch. On the other hand, patching software from upstream takes seconds to do (not counting the unavoidable build time).

Downside: In order to accomplish the upside, it completely changes the filesystem layout. "./configure; make; sudo make install" is basically guaranteed to not work. To install software not in nix, you will be writing a nix expression. There is a command to create an FHS compatible namespace that works for e.g. Steam, but applications installed in there are second-class citizens, so it's preferred to not do this for anything that can simply be installed via a Nix expression.

ComputerGuru4• 9 years ago

Have you tried FreeBSD before? It ships with an awesome binary package manager and a virtual tree of sources for all the packages in the package manager at the same time. You can install the binary (sudo pkg install foo) or cd into the source dir, fetch the sources, patch on a whim, then “make” (even if the project upstream does not use make) and install.

honr2• 9 years ago

As someone who hasn't actively used nixos (yet), only nix (the package manager) for a while:

I have always been curious and dumbfounded why he (Eelco) abandoned Maak (his build system, that I thought, would go together with nix). I think a proper, integrated build system (integrated with nix, the package manager) is a (or the?) missing piece to improve software development.

alsadi• 9 years ago

Y U no ostree?

https://ostree.readthedocs.io/en/latest/manual/introduction/

avinassh• 9 years ago

Any reason why the GitHub repos are under an user account but not an org?

[0] https://github.com/gittup

juanmirocks• 9 years ago

Good idea. Reminds me of hombrew in macOS

mrmr19933• 9 years ago

It seems like the distro name may violate the Git trademark policy [0]. From [1]:

... you can't use the mark in a way that implies association with or endorsement by the Git project.

It might be worth checking with the Git project to see if they would approve this use or find it within their allowed uses.

Shoothe• 9 years ago

I agree. Usually names in format XY where X is a trademark are considered infringing. See several cases of projects named docker-something or Windows Subsystem for Linux, a name that technically doesn't make sense but "for X" names are usually OK.

bmarkovic• 9 years ago

Er.. Github, gitlab, gitkraken.. I don't think having git in the name is enough to violate that clause

eat_veggies• 9 years ago

Those all got written permission to use the name.

See a similar discussion from a few days ago: https://news.ycombinator.com/item?id=15991785

jwilk4• 9 years ago

404 Footnote Not Found

mrmr19933• 9 years ago

Looks like the gremlins ate them, sorry!

[0] https://git-scm.com/trademark [1]: https://public-inbox.org/git/20170202022655.2jwvudhvo4hmueaw...

trendia• 9 years ago

Does GitHub have to pay a fee to use Git in their name?

mrmr19933• 9 years ago

From https://public-inbox.org/git/20170202022655.2jwvudhvo4hmueaw...

The USPTO initially rejected our application as confusingly similar to the existing trademark on GitHub, which was filed in 2008. While one might imagine where the "Git" in GitHub comes from, by the time we applied to the USPTO, both marks had been widely used in parallel for years. So we worked out an agreement with GitHub which basically says "we are mutually OK with the other trademark existing".

beejiu• 9 years ago

Is this style of package management not what (broadly speaking) differentiates BSD from Linux?

krylon2• 9 years ago

All the BSD systems[1] have had binary packages (at least optionally) for a while now. On OpenBSD, while the ports tree is still there when needed, the recommended way is to use binary packages.

[1] I am not entirely certain what the situation on DragonflyBSD is, I have not touched it in a long time.

wallnuss• 9 years ago

My favourite sentence is: Note: I've never actually tried this, so we'll just assume it works.

mirceal• 9 years ago

stopped reading at modules. I think this is an interesting ideas, but if one would seriously consider this, this is a disaster waiting to happen.

HumanDrivenDev• 9 years ago

Sounds good - can it run a decent VCS though, like mercurial?

itomato• 9 years ago

This is interesting. I have been wondering about the viability and communuty interest in a service to enable subscription service for CI/CD systems at the project level.

OSS maintainers don't have a unified mechanism to alert packagers of updates.

In place of a dirty hack to power IFTTT and Jenkins from fetchmail scripts when there is an announcement, we could have intelligence and transparent automation at the distro-level.

Maintainer patches a 0-day, and there's a mad scramble for everyone to patch, test, package and publish.

I would rather not wait on Debian to release a distribution or be forced to cherry-pick from Sid.

OpenSolaris and OpenBSD could have feature parity with Cygwin and Alpine.

We could have concurrent versioning, automation and container and filesystem-based deployments become a little less risky.

Less focus on Vendor, more on Developer.

icebraining• 9 years ago

Debian has vcswatch to keep track of updates. I think it only works if the upstream repository keeps the Debian control files updated, but keeping track of commits that update a generic CHANGELOG file should be simple.

That said, I don't think the vendor can be avoided unless you want to live on the edge. If you're on a released version of a distro (otherwise, you wouldn't have to cherry-pick from Sid), chances are you're not on the latest version of the software, and the patch has to be backported by the distro maintainer to version you are actually on.

andrey_utkin2• 9 years ago

OSS maintainers don't have a unified mechanism to alert packagers of updates.

Gentoo developers have http://euscan.gentooexperimental.org/ . Quite some automation is on the way to come soon.

jwilk4• 9 years ago

I couldn't find actual version control repositories for ncurses

https://github.com/ThomasDickey/ncurses-snapshots

This repo is generated by exporting from RCS(!):

http://invisible-island.net/personal/git-exports.html

There's also an unofficial git repo, which is more useful IMO:

http://ncurses.scripts.mit.edu/?p=ncurses.git;a=summary

jwilk4• 9 years ago

I couldn't find actual version control repositories for [...] nethack

https://github.com/NetHack/NetHack

undoware• 9 years ago

So.. 'tup' means something, uh, specific, in certain vernacular Englishes.

JepZ• 9 years ago

To me this looks very similar to running a source based distro (Gentoo,Exherbo,...) with lots of packages using git as a fetch method (Gentoo uses the -9999 version suffix, Exherbo calls them scm packages).

Can anybody tell me when I should use gittup instead? Is it just a matter of size? Is rebuilding significantly faster with gittup due to the automatic rebuilding?

I mean to me it looks that gittup has some serious limitations in its current state (e.g. no 'make') while the others are full blown linux systems.

chenglou• 9 years ago

For a build system similar to Tup, check out http://physics.oregonstate.edu/~roundyd/fac/introducing-fac....

Explicitly inspired by the former, same build paradigm, doesn't use FUSE. Has some neat git integrations. Doesn't work on macOS/Windows yet though.

One funny feature that falls from the design is that you can just keep running & failing a fac build, and it'll nudge you toward the correct build.

Too• 9 years ago

Isn't this similar to yoctos bitbake? A bunch of package repos with a small build combiner on top.

I'm also curious how else would you do it? The article make it sound unique but this seems like the obvious solution, at least conceptually. How do other distros manage and build their packages?

Gittup, a Linux distro on top of Git

Related Stories