Who helps your Linux distribution run smoothly? Thank a packager today

http://opensource.com/business/14/2/thank-a-linux-packager-today

by Tsiolkovsky • 12 years ago

215 65 12 years ago

opensource.com

keenerd2• 12 years ago

Hey Arch-folk! Top requests for AUR packages you'd like to see brought into the [community] repository?

edit: Or previously unavailable software that should be in the AUR?

616c2• 12 years ago

Just keep doing what you guys are doing (started using Arch after years of Debian, Fedora, Ubuntu, in order of preference). I am hooked. Thanks for finding the sweetspot between things I like with FreeBSD/Gentoo/Debian in terms of packaging and philosophy.

agumonkey2• 12 years ago

hehe, came here to thank the arch team[s], I've been smooth sailing for a few years on top of their creation.

bretthoerner2• 12 years ago

I just looked at my AUR list and most are non-free. A couple popular standouts that are free: shutter, vagrant, gradle, leiningen, apache-ivy, jq

arunc• 12 years ago

Another archer here, since the 2009.

sentenza2• 12 years ago

I'd add bittorrent-sync. Encrypted, de-centralized Dropbox replacement.

dontdieych• 12 years ago

shutter +1

zhemao• 12 years ago

The arduino package has been sitting in the AUR for an oddly long time given its relative popularity. I've been waiting for it to go into [community] but it hasn't so far.

zanny• 12 years ago

Heres basically all the free "small" (<10MB~) packages I could find in my aur installs:

  android-sdk-platform-tools
  archlinux-artwork
  audiothumbs
  bup
  darkplaces
  dolphin-box-plugin(-git)
  eduke32
  firefox-aurora
  firefox-beta(-bin)
  firefox-nightly
  fontconfig-infinality
  freedoom
  freetype2-infinality
  glmark2
  google-talkplugin
  grive
  gzdoom
  hexen2
  kcm-ufw
  kcmsystemd
  kde-servicemenus-imgur
  kde-thumbnailer-epub
  kdeconnect
  kdeplasma-applets-veromix
  kup
  lib32-sdl_mixer
  lib32-simplescreenrecorder
  lib32-smpeg
  libjpeg6
  libx11-ldpreloadnograb
  mprime
  nwn-gog
  openoffice-thumbnail-plugin
  php-dio
  plasma-mediacenter
  qgears2
  qt5-jsbackend
  simplescreenrecorder
  speedtest-cli
  tor-browser-en
  transfusion
  ttf-neverwinter
  ubuntu-mono
  yamagi-quake2
  zandronum

These packages are also in that group, but all have huge package sizes so I imagine that would contribute to their exclusion (mostly due to bundling image or sound data):

  android-ndk
  android-sdk
  caledonia-bundle
  eduke32-dukeplus
  humanity-icon-theme
  kdeartwork-sounds
  kdeplasma-applets-starfield-wallpaper
  ksplash-archpaint2
  light-themes
  openarena
  qt5-examples
  wallpaper-archpaint2

These are proprietary, or are only used by proprietary software, but to me seem to fall in the same class as Steam / Flash / etc, in that they are all at least freeware.

  binkplayer
  btsync
  btsync-autoconfig
  desura
  dropbox
  hon
  honpurple
  minecraft
  minecraft-server
  spideroak

"Top" would be most of the kde stuff and all the foss id-tech engines for games. Though I imagine the reason none of them are in is due to some licensing reasons concerning the game data, though it is worth mentioning the demo data for most of those games is also in the AUR.

And simplescreenrecorder. Best desktop screencasting software I've ever used, wondering why its in the AUR.

nextos• 12 years ago

Thanks for the awesome work to all Arch packagers.

I really think isync deserves being brought to [community]. It's an awesome IMAP syncing tool created by the same guy who coded mutt and used by a lot of people.

It really deserves moving to [community] to get to a wider audience. IMHO it's much better than offlineimap.

chimeracoder• 12 years ago

git-annex would be great!

I've come to depend on it heavily for managing all my media files (pictures, music, video, etc.).

I currently maintain the git-annex-bin package in the AUR[0], which uses the official binary, but that's compiled for Debian and has issues on Arch.

For people who want to use git-annex on Arch, the choices are either (1) use this buggy binary, or (2) install cabal and a bunch of Haskell stuff[1], just to compile from source.

[0] https://aur.archlinux.org/packages/git-annex-bin/

[1] which requires adding a separate repository, since Haskell is no longer in the main repos, and also installing a bunch of Haskell libraries that aren't in the AUR.

616c2• 12 years ago

It is my experience with Haskell Arch packages, as the lack of forum feedback suggested to me, with my troubles with dependencies for pandoc or git-annex-bin,[0] to go and cabal it.[1] That is just me.

I know that goes against the point of this whole post, but I wanted to learn more about Haskell and part of that for me was leaving the farm and running wild, so to speak.

[0] https://aur.archlinux.org/packages/haskell-setenv

[1]https://bbs.archlinux.org/viewtopic.php?id=173781

viraptor• 12 years ago

Vagrant!

ginko• 12 years ago

It would be great if glfw2 were in community.

While I like Arch's philosophy of always being on the cutting edge and making the glfw package be GLFW 3.0, I think it would be good if GLFW 2.x stayed in the official repositories for a while as a separate package. The two packages don't interfere with each other. It's basically the same situation as with SDL 2 and 3.

gillianseed• 12 years ago

How about 'packjpg' for AUR, it's an open source lossless JPEG compressor which gives ~20-25% compression, the resulting compressed files are either stored as single (.pjg) files or as archives (.pja) which can then be decompressed back into the original jpeg files.

http://www.elektronik.htw-aalen.de/packjpg/

aw3c2• 12 years ago

aarddict as offline wiki reader

gpa, the gnu privacy assistant

qgis is free GIS

gimagereader is nice OCR

zdw2• 12 years ago

If you're a developer, try to package your own things for the distros you care about. And accept patches that fix packaging issues. Think about how much easier your ops friends, and the continuous integration systems and similar will have it if all they have to do is run an "apt get" or "port install" to get a fully built version of your code.

Things that help:

http://fedoraproject.org/wiki/How_to_create_an_RPM_package

https://wiki.debian.org/IntroDebianPackaging

http://guide.macports.org/chunked/development.creating-portf...

http://www.openbsd.org/faq/ports/guide.html

http://www.netbsd.org/docs/pkgsrc/creating.html

massysett• 12 years ago

Shouldn't packaging be left to each distributor's experts? As a developer of software you're an expert in whatever your software does--web server, calculator, compiler, whatever. There is no way you can be an expert in your software's functionality and in packaging, especially for more than one distribution.

I'm always wary of installing packages not provided by my distribution, just because I have more faith that my distribution's packagers know how to package and they are more likely to use QA processes that will catch packaging bugs. I'd rather compile from source than install third-party packages; I can make my own rudimentary package that would be woefully inadequate for general use but that is perfectly sufficient for my needs.

blueblob• 12 years ago

I think that depends. ArchLinux packages are almost just bash scripts. People can request fixes. I am sure when you first started programming you weren't an expert and that same logic applies to making packages. Multiple distributions you may be right.

I suppose it is fair enough that you would want to install third party packages from source but with most package managers I would guess you could download the package and look at how it is built or at least what files it contains. Using packages can make cleaning up a bad package easier.

zdw2• 12 years ago

Shouldn't packaging be left to each distributor's experts?

My take is that any package is better than no package. At least then there's something for a sysadmin, distributor, or other ops person to work with, and it shows that you actively want your software to be packaged.

You're the expert on your program and what it needs to work, knowing what the dependencies are, etc - inject that knowledge into the spec/port file and then whatever upstream packager will have even more to work from.

I can make my own rudimentary package that would be woefully inadequate for general use but that is perfectly sufficient for my needs.

Why not make that and share it? Having the rudiments around as a starting point to fix and improve is almost always better than having nothing.

Think of making packages as a collaboration, rather than something that someone else has to do for you.

nailer• 12 years ago

Shouldn't packaging be left to each distributor's experts?

I get where you're coming from - there's some apps that want to redefine how packaging or your OS works - but I think it's easier for software devs to learn things like the FHS and packaging than it is for distro maintainers to judge whether an arbitrary piece of software works on not.

+1 making your own packages though. More people should do that - if you're smart enough to compile from source, you can easily make an RPM or Dpkg.

forgottenpass• 12 years ago

There is no way you can be an expert in your software's functionality and in packaging, especially for more than one distribution.

The goal of packaging your own software shouldn't be to spin up a public repository of your own, but to get it into distribution repositories. You don't need to be an expert to make decent packages, nor do you have to toil on your own for perfection. Just have something to show for yourself when you get the package reviewed by the distribution maintainers.

bostik3• 12 years ago

Debian packaging has a long history of being insanely hard to get right. That was certainly true in debhelper v5 and v6 times - if you wanted to package ANY software, you needed to copy-paste from an existing debian/rules and then tweak all the dh_* calls until you got a working build. It was possible to get the packaging into a "working condition" by sheer stubbornness. (Trying to learn Debian packaging from scratch was like trying to learn auto* from scratch.)

With debhelper v9, and it's lovely sequencer things are a lot easier. The once-so-arcane debian/rules is now only a few lines, and as long as you set DH_VERBOSE=1, you can see exactly which internal helper gets called at any point of the packaging run. If you need to override a step, simply define "override_dh_auto_FOO" target and that's it.

There's less boilerplate, and thus the packaging will be quite a bit easier to adapt to official Debian standards. So it will be (hopefully) less work for an official NM/DD to adapt your simple rules and get the package into a shape where it could be uploaded to experimental or unstable.

Back in my previous life I actually wrote two or three instruction sets for random developers on how to get their software into a working debian package. And then, when the customer project manager wanted to try his hand at coding too, I came up with a bunch of reductionist scripts and templates that allowed to automatically generate a sensible debian/* hierarchy. (That effort was later taken over by developers more capable than me - and made into a proper QtCreator plugin. That thing has supported "easy debian package" system for a couple of years now...)

I still can't produce packages that would pass the extremely high bar Debian sets for their packaging standards, but I sure can put together a packaging that works and is relatively easy to polish.

xioxox• 12 years ago

Yes, I maintain Fedora and Debian packages for my upstream project (I'm not a Debian developer - I pass my package to a sponsor).

Debian is much harder to get right, though I do have more experience maintaining Fedora systems. The Debian packaging process seems an order of magnitude more difficult to get right, mainly because of the magic documented in various scattered documents and the many different build systems. I've yet to find a good document explaining how all this stuff interacts with each other. rpmbuild is much more conceptually easy to understand for me.

You also have more boilerplate documents (e.g. copyright), that Fedora doesn't care about. Fedora also has a nice system where you can get builds done on a server with various architectures and for different releases.

vacri• 12 years ago

I found it difficult to figure out .deb packaging for our in-house services when I had a crack at it last year. There are a dozen ways to do it, with no real agreement found by my google-fu, and if you don't have a makefile, it gets even harder to find documentation to help.

I saw a comment on a feature request in FPM by Jordan Sissel: "I'd think about implementing it, but there's so much silly ceremony around .deb packaging" (paraphrased). 'Silly ceremony' stuck with me. I'm sure it makes sense once you know, but there's a steep learning curve.

takluyver• 12 years ago

Having tried to do some Debian packaging, it still feels horribly arcane and hard to get right. I think part of the explanation lies in "If you need to override a step, simply define...". How do I know what steps there are, and what each one is doing when I don't override it?

The system saves on boilerplate, but it doesn't provide new abstractions for the packaging process. So you still have to know what is actually going on, and that's harder to discover when it's being done magically behind the scenes than when it's done by boilerplate in the rules file. So it's easier for experts now, but hard for novices to get beyond the most trivial cases.

Of course, the impression that it's difficult is reinforced by Debian's extremely strict standards for accepting packages.

peterwwillis2• 12 years ago

In an ideal world, software developers should merely prepare their software to make it easier for packagers to whip up a package based on a template. As a packager I know the pain of having to discover the magical way a piece of code is intended to be built, tracking down its dependencies, and stringing together the means of packaging the finished product based on how the developer assembled it.

Some of the problems you'll face when packaging random code:

  * non-standard version numbers
  * non-standard file paths
  * non-standard configure/make/install routines (or none at all)
  * custom or non-standard build/install tools
  * lack of build/configure/install documentation
  * lack of dependency documentation (including what versions of what dependencies might conflict)
  * code that isn't portable past the desktop the developer wrote it on
  * lack of best practices in setting up multi-user permissions
  * lack of pre/post install scripts
  * lack of simple init scripts
  * lack of minimal configuration files
  * lack of pidfile, log rotation, crontab, environment, and other necessary ops functions
  * patching code just to get them to build and install on your distro
  * bundled, highly patched, third-party software (!!!)

Learning to make packages is easy, but what's even easier is leaving your source in a state that's convenient for packagers of all distros to put it together with little or no effort. Here's some suggestions on how to do this:

1. Make sure your code is portable. Please, try not to hardcode things; use a 'configure'-type process to allow someone to override things like file paths, usernames, and other variable data. Test it on multiple architectures or distributions to see what kind of dependencies you might need to document.

2. PLEASE use a standard configure/make/make install method for your language, and don't write one yourself or depend on some backwards antiquated build tool that nobody uses. Interpreted languages often have their own standard, and compiled languages often use autoconf/automake. Learn them and use them, or at the very least, make an incredibly basic Makefile. And please document the expected permissions and users (chowns/chgrps/chmods).

3. Make sure that you can run one command for each of the configure/make/make install steps. Make sure it can be passed options or environment variables to change the root directory to install into. Make sure your install procedure puts them all in Filesystem Hierarchy Standard paths. Make sure it works as a non-root user; the packager will add the permissions and ownership as metadata to the package.

4. Provide working samples of all the things your app will need. Crontabs, log rotation, pid file tracking for daemons, configuration files, documentation, init scripts, preinstall/postinstall/uninstall scripts.

5. For the love of god, use standard version numbers. Either X.XXXX, X.XX.XX, or X.X.X.X. Pick one and stick with it. Otherwise different distros might actually use different version schemes for their packages which confuses everyone. Please don't include any non-numeric characters (fuck you, openssl! hex doesn't belong in versions!!!)

6. Fully document your ChangeLogs. A packager needs to know that your new release now requires a newer dependency or breaks ABI compatibility, because now they have to modify the metadata of the package to specify what deps will or won't work. Remember that users often install and uninstall different versions of different software.

angersock2• 12 years ago

"2. PLEASE use a standard configure/make/make install method for your language, and don't write one yourself or depend on some backwards antiquated build tool that nobody uses. Interpreted languages often have their own standard, and compiled languages often use autoconf/automake. Learn them and use them, or at the very least, make an incredibly basic Makefile. And please document the expected permissions and users (chowns/chgrps/chmods)."

This, a thousand times this.

A basic Makefile is always preferable to me finding a problem with your magical unicorn make system of choice (read: CMake).

endgame• 12 years ago

Given the GNU makefile conventions ( https://www.gnu.org/prep/standards/html_node/Makefile-Conven... ), I'd say that packaging with automake is preferable to hand-rolling a Makefile.

If you don't know automake, this tutorial is fantastic: https://www.lrde.epita.fr/~adl/autotools.html

grosskur2• 12 years ago

This is exactly why I created Package Lab---to make Linux packaging easier for developers and ops folks. Currently in private beta and would love feedback. Please get in touch with me if you're interested (email in profile).

https://packagelab.com/

lwhalen• 12 years ago

aaaand invite request sent. I'm a big believer in packaging, and $current_job is of the opinion that packages "aren't Cloudy/DevOps/new enough" to be worth investing in. If I could show them a RESTful API, that might just be what it takes for them to move beyond the pain of 'git pull' deploys!

Mikeb85• 12 years ago

To add to this, OpenSUSE build service is invaluable, and not just for SUSE distros. https://build.opensuse.org/

znowi• 12 years ago

I'd like to have a honorable mention of Archlinux here. Arguably, it has the easiest package management out there.

To create a package you write a single PKGBUILD file with a few required fields (like pkgname, pkgver, etc) and a bash build script - that's it. Then you run "makepkg" and it's done.

Take a look for yourself. This is a build script for tmux:

https://projects.archlinux.org/svntogit/community.git/tree/t...

If you know of an app which isn't available for Arch yet, I welcome you to give it a try and build a package. It's really easy.

And for completeness - the package guidelines for Archlinux:

https://wiki.archlinux.org/index.php/Arch_Packaging_Standard...

stefantalpalaru• 12 years ago

Gentoo's build script for tmux has more tricks up its sleeve: http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/app-...

patcon• 12 years ago

Or ask them to join Gittip! After they're signed up, others will be able to start giving them recurring weekly gifts that are totally anonymous and categorically no-strings-attached. It's like a DFTBA grant.

Disclaimer: I've taken 4 months off work to help work on the platform.

rdtsc• 12 years ago

This cannot be overstated, I have heard during conferences packagers being made fun of, something to the effect of "they are so cute, they don't understand our problems".

To me it seem some are stuck between a rock and a hard place -- distro requirements vs application specific (vendor) requirements.

So vendor likes to bundle specific library .so in the package or compiles external modules as static. Distro requirement, for security reasons for example, disallow that. Packager is left in the middle trying to reconcile the differences between the two. I have seen people stop maintaining their packages.

Yeah sometimes there is nothing you can do and the latest coolest database or web browser or game just can't easily go into the main distro.

But also I can't tell you how happy I have been in the past if a library I wanted to use was in the repo and I could just to an apt-get or yum install and all that is due to someone putting the work into doing that. Now instead of packaging it myself, building it, I saved a lot of time.

eplumlee• 12 years ago

Don't just thank them. If you cannot support with your time, send them new hardware and beer.

abruzzi• 12 years ago

I thank my packager by paying them ~$12k a year for "support".

xfalcox• 12 years ago

After being forced to use a non Debian distro at new servers at work (SLES) I am so proud of Ubuntu packaging community.

TallGuyShort• 12 years ago

Debian does have an impressive list of packages, but I'd rather use Zypper than Apt any day. It's the only mainstream package manager that actually lets you choose what to do when a dependency can't be satisfied or there's a version mismatch.

caycep• 12 years ago

could they have used a better example than MUMPS tho....yes, I know it works with its peculiar brand of duct tape, chewing gum and baling wire, but still...

jude-• 12 years ago

I'm lazy when it comes to these things, so I use FPM. It's great if you're dealing with RPM and .deb packages (and a few others). I saw someone else (vacri) mention it in this thread, but it definitely deserves consideration if you want to side-step setting up a packaging environment.

Link: https://github.com/jordansissel/fpm

ksk• 12 years ago

To me, its rather sad that we still have to rely on multiple third party apps for dependency management - something that a modern OS should have baked into its core branch.

Who helps your Linux distribution run smoothly? Thank a packager today

Related Stories