Install Debian packages without starting daemons

http://major.io/2014/06/26/install-debian-packages-without-starting-daemons/

by liotier • 12 years ago

58 40 12 years ago

mercurial• 12 years ago

Surely not all daemons start automatically. Some of them source files from /etc/default, with an "off" setting per default. And there are plenty of daemons for which it makes sense to start automatically (eg, installing pulseaudio and expecting it to work without having to figure out how to configure it is not an unreasonable expectation). So maybe:

- more daemons (eg, puppet) ought to be "off" per default when it doesn't make sense

- apt-get should provide a switch to prevent newly-added daemons from starting, which could be used by provisioning systems

But saying that "daemons should never start at install" is silly.

duskwuff• 12 years ago

The latter is precisely what the /usr/sbin/policy-rc.d business that this post is talking about is for.

erikano• 12 years ago

This appears to me to be correct. Found a file /usr/share/doc/sysv-rc/README.policy-rc.d.gz on my system where it says:

POLICY-RC.D Policy layer (/usr/sbin/policy-rc.d) interface:
Most Debian systems will not have this script as the need for a policy layer is not very common. Most people using chroot jails just need an one-line script which returns an exit status of 101 as the jailed /usr/sbin/policy-rc.d script.

I think, however, that the point that the parent comment was trying to make was that it'd be convenient if this was a switch to apt-get. After all, if you are just looking to install one additional daemon on a running system, it'd be a whole lot easier if you could:

# apt-get install --no-start-daemon example

than to have to activate a policy script which you might forget to remove or deactivate.

However, in the file I mentioned above, the next line says:

The /usr/sbin/policy-rc.d file must be managed through the alternatives system (/usr/sbin/update-alternatives) by any packages providing it.

So maybe one solution is to create your own script to manage it.

In fact, your script could even be a wrapper for apt-get. To be totally transparent in day-to-day use, you could then make apt-get be an alias for your wrapper script.

Your wrapper then:

1. Activates the policy.

2. Calls the real apt-get.

3. Deactivates the policy.

serf• 12 years ago

"daemons without sane defaults should never start at install"

It's silly that certain daemons check for a default settings file to decide if an execution is permitted rather than skipping that logic and defaulting the service off until either user intervention provides sane operating defaults, or the defaults are known to be sane otherwise.

I agree that a switch should be included with the package managers, but I also believe that the package managers should look for a switch from the package maintainers as well.

"areDefaultsSane = 0" should require user intervention to start after install and would be intended for things like insecure network daemons or daemons that control thermals/power/otherwiseImportantThings, whereas things that flagged "areDefaultsSane" as true would be started without intervention, this being intended for things like pulse.

things like thinkfan (a cpu/fan manager for linux) include "run_at_start = False" in the /etc/default file and things like that, but it feels woefully inadequate to rely on the package itself to check for something potentially mean when the environment could handle it in a standardized way, perhaps within whatever metadata is included in the deb file rather than the packages' included /etc/default/? (disclaimer: I don't know how deb files are put together)

icebraining• 12 years ago

A file in /etc/default is a standardized way. It's part of the Debian Policy.

Putting it in the metadata wouldn't help, since the point is to be mutable, that is, to allow the admin to switch the default to "run on startup".

stormbrew• 12 years ago

AFAIK it is Debian (and Ubuntu) policy for all packages to auto start services. And it is also quite difficult to pre-set package config files before installing, especially if you want to script it with a configuration manager like puppet. I can't recall ever seeing a package in the official repositories for which this is not the case, and it is a real and genuine pain point in using Debian as a server.

akurilin• 12 years ago

Would this be a good fit for preventing Postgres from starting on Ubuntu? In the past with ansible I've had to install it, stop it, move the DB files to the non-default location of my choice, change the configs, and start it up again. Would have been easier to skip a couple of those steps.

claudius• 12 years ago

Unless you were happy with the default configuration, in which case it is much easier not having to start the thing manually.

pbhjpbhj• 12 years ago

What's the current Ubuntu equivalent of services.msc ?

mw42• 12 years ago

I had the same issue. There is another solution for postgres.

Install postgresql-common first. Edit /etc/postresql-common/createcluster.conf to disable creation of the cluster on install (or change the options used to create the cluster) and then install postgresql-x.y

pling• 12 years ago

This is one thing that has always annoyed me about debian. Nice to see there is a solution.

I rather prefer the /etc/rc.conf approach of BSD systems though.

nnnnni• 12 years ago

It's actually why I left Debian for another distribution... I think that it may have also had a bad habit of overwriting /boot/grub/menu.lst every time that program was updated. That could have been Ubuntu, though. It was a dual-booter's nightmare! I had to boot from a live cd and fix way too often.

bildung• 12 years ago

I think your menu.lst problem is neither Debian nor Ubuntu, but a change in management of that file by grub itself. Starting from grub 2.0 /boot/grub/menu.lst gets created from scratch on every update of grup, kernel or init.rd, using the parameters given in /etc/grub.d/ (in Debian). If you want to dual boot to another os, you have to place your commands there, and they'll find their way into menu.lst automatically on every kernel update.

That bit me, too, btw :)

dale-cooper• 12 years ago

I think a saner default would be to not start daemons on install. It's one of the reasons CentOS has grown on me.

The biggest annoyance is the lack of consistency. Some daemons start on install, others do not. Some use an /etc/default file to control it, others do not.

atoponce4• 12 years ago

The /etc/default/<daemon> configuration file comes from upstream, and is not created by the package maintainer.

SkyMarshal4• 12 years ago

Another alternative is just bypass the Deb repo's altogether and install from source, but do so using the awesome Debian Alternatives tool [1] so as not to drop a bunch of unmanaged files into /usr/bin, /usr/local/bin, etc. Takes some setup work, but very powerful and useful after you get it working. Example scripts:

https://github.com/byrongibson/scripts/tree/master/install/h...

https://github.com/byrongibson/scripts/tree/master/install/j...

https://github.com/byrongibson/scripts/tree/master/install/s...

Can do similarly with any other package. Download and compile the source to --prefix=/opt/name/version, look at the /opt structure (it will mirror /usr/bin, /usr/local/bin/, /usr/share/man) and script the update-alternatives command correspondingly.

Tons of benefits, among them being better post-install startup control, but also you can install multiple versions side-by-side and swap among with a single command. Upgrade painlessly, rollback if the new version breaks things, all very easily.

Reminiscent of Gobo Linux [2] but still a Debian.

[1] https://wiki.debian.org/DebianAlternatives

[2] https://en.wikipedia.org/wiki/GoboLinux

ars3• 12 years ago

If you do that then you lose the automatic package updates, for example for a security update. And that's one of the best parts of Debian.

atoponce4• 12 years ago

Installing packages from source is fine, but then you have to maintain all security updates and patches from upstream, and you need full control over where all the files are populated on your filesystem. This is a lot of administration overhead. It "works", but it's also work. I rarely install software outside of the APT system, for this very reason.

SkyMarshal4• 12 years ago

True, but for your primary tech stack you should be on top of that anyway, and the benefit is you can install the full updated version beside the old version, swap over to the updated version with a command, test your app, and if there are any regressions, roll back to the old version with a command.

Anything not part of your primary stack, sure use the repo.

cdjk• 12 years ago

I've used stow to manage installs under /use/local before, which seems like a slightly easier approach. It does something similar but manages the install as a series of symlinks from /usr/local/bin to /usr/local/stow/<packagename>/bin.

It's pretty easy to setup and works the same on any distro.

SkyMarshal4• 12 years ago

I tried stow a while back too but there was something I didn't like about it, but can't remember. It might have been that stow only supports a single version of each package or something, less flexibility in putting version no in the paths, maybe?

emanuelez• 12 years ago

I've been bitten myself by this during my Chef days. Good to see a solution!

ars3• 12 years ago

If you don't start a daemon on install then it should also not start on reboot.

They go hand in hand.

So this really has nothing to do with install, but with the default configuration of a freshly installed daemon: Is it by default set to start or not.

mtdewcmu• 12 years ago

I agree. If the install is finished, then the system should be in its new normal condition, and that condition is either daemon-on or daemon-off. There is quite likely a real issue here worth fixing, but the fix is probably more nuanced than just daemon-off.

viraptor• 12 years ago

There are good reasons to prevent the auto-start behaviour. Having your own configuration ready for example. Preventing things from breaking when something is already listening on their default port. Preventing creation of the initial files in the wrong place.

You can find many more cases if you do more server automation. Most people don't need them, but it can be very useful. And no, it's got nothing to do with start on (re)boot.

ars3• 12 years ago

There are good reasons to prevent the auto-start behaviour. Having your own configuration ready for example.

Certainly. But in that case the daemon should also not start on reboot until configured.

After an install the system should be in the same state as after a reboot - there should not be an "in between state" after an install and before a reboot because such a state is not persistent.

There are tons of daemons that require that they be enabled before starting.

And no, it's got nothing to do with start on (re)boot.

You are (I hope) confusing start as in attempt to run on reboot, and actually successfully run. Set the daemon to start on reboot, then check the config file and immediately quit since the file is not configured.

This is extremely common in debian installations.

There are also tons of deamons that feel that their defaults are good enough, and start right away.

But the thing in the article about "install" not meaning "run" is a red herring.

Confusion• 12 years ago

Why must they go hand in hand? What goes wrong if after installing a package the daemon is not started automatically, while the configuration is such that it will start on reboot?

rlpb• 12 years ago

This mechanism has existed for a long time. It's a shame that puppet and chef do not use it by default.

regularfry3• 12 years ago

Learning this seems to be a rite of passage for anyone who spends enough time doing Debian deployment work.

I think the complaints that starting daemons should not be the default are missing the mark: Debian is there to provide a complete, working system on its own terms. They want you to be able to apt-get install <thing> and have <thing> working, out of the box, with no further work on your part. I get that there are situations where this is inconvenient, and for my money there ought to be an `apt-get install --no-launch` switch rather than monkeying with an obscurely-named shell script, but if you've run into this then you've run into an instance where the system wasn't designed for you.

This is the same reason you probably shouldn't, for instance, do development against the packaged ruby - it's there to run the applications which Debian provides, no more, no less. If your needs align with the needs of the end-user applications which Debian provides, that's great. As soon as you step outside those lines, you're going to have a bad time.

TallGuyShort• 12 years ago

Agreed. I've run into a similar kind of problem with their package manager as well. If you try to install a package for which some of the dependencies can no longer be met because of a poorly maintained repository or something, you make take a closer look at the application and determine that you would be able to do the work you needed to do without that dependency being met. All other major Linux distros have package managers that allow you to override the dependency check (in the case of SUSE, you can do this interactively with excellent granularity, on RHEL you can simply disable the check altogether but then you're on your own). On Debian and Ubuntu, you have to switch to using dpkg to install the package directly instead of apt in which case you can get past the dependency check, but then as soon as you use apt again it will refuse to work until you've removed the offending package or satisfied the dependency.

I understand the need to warn the user about the consequences of what they're doing, but at the end of the day they are using the software, not the other way around. As the admin, I want the ability to override checks once I have done my due diligence.

wazoox• 12 years ago

They want you to be able to apt-get install <thing> and have <thing> working, out of the box, with no further work on your part.

Except that there is practically no way for say, winbind, or openldap, or bind to be able to start up successfully without configuration first. Then the oh so frequent case nowadays of preparing a deployment image in a chroot and bang, install fails because of unwanted starting services that conflict with the host.

This is the same reason you probably shouldn't, for instance, do development against the packaged ruby

Excuse me, this is insane. You're using a well-maintained distribution because you don't want to manage your own installation of Ruby or Python or Perl or PHP and countless modules, frameworks and the like, on the production systems. All the point of running Debian is developing against the provided binaries, because you want to use the same versions in dev and deployment, don't you??? and have an easy maintenance, upgrades, etc.

What is needed is a simple switch to apt-get/aptitude/dpkg to NOT start the services when installing, and maybe some other switches to NOT run preinst/postinst. That's it.

regularfry3• 12 years ago

Except that there is practically no way for say, winbind, or openldap, or bind to be able to start up successfully without configuration first.

If you know the configuration you want, you can write the configuration files first. `DEBIAN_FRONTEND=noninteractive apt-get install <foo>` won't (or shouldn't) overwrite pre-existing config files.

Then the oh so frequent case nowadays of preparing a deployment image in a chroot and bang, install fails because of unwanted starting services that conflict with the host.

What I was saying above about the system not being designed for you? This is exactly that. I'm not saying this isn't a valid thing to want to do, just that the system isn't explicitly designed to support this use. It is unsurprising that there are rough edges.

I wouldn't be at all surprised if someone comes up with a container-based approach which makes this just work, but I've not seen it done yet.

Excuse me, this is insane. You're using a well-maintained distribution because you don't want to manage your own installation of Ruby or Python or Perl or PHP and countless modules, frameworks and the like, on the production systems.

No, I'm using a well-maintained distribution so that I don't have to take control of anything I don't want to care about. I know that the system outside what I'm directly responsible for is stable. No surprises.

I do want to care about my runtime. I've found it makes more sense to treat the runtime as part of the application I'm working on rather than rely on the distro-supplied packages because the distro's goals are simply divergent from my own. They want absolute stability; I might want to upgrade a patch level so I can take advantage of improvements in GC, say.

This is potentially more true of ruby than, say, perl. Going back to my earlier point, I think this is because there have historically been more apps in Debian implemented in perl, so whatever you're doing, your needs are more likely to overlap with what Debian already provides for itself. The same goes for PHP and Python, to an extent.

All the point of running Debian is developing against the provided binaries, because you want to use the same versions in dev and deployment, don't you???

You don't have to use the Debian-supplied ruby to know that you're running the same version in development and production. Personally, I use the Debian-supplied infrastructure tools to build my own ruby packages, and to provide a repository which both dev and production can pull from.

The bonus from this is that I don't need to do a big-bang upgrade of my app every time Debian roll out a new stable release because they've changed the runtime version and all the associated packages.

What is needed is a simple switch to apt-get/aptitude/dpkg to NOT start the services when installing, and maybe some other switches to NOT run preinst/postinst. That's it.

    apt-get install --recon apache2 | grep ^Inst | awk '{print $2}' | while read pkg; do apt-get download $pkg; dpkg-deb --extract ${pkg}_*.deb; done

...will unpack apache2 and its dependencies without running pre or postinst scripts. Fiddly, but it's doable. Now, for extra fun, you get to check each of the dependencies and find out which pre and postinst scripts you did want to run after all.

I agree that this should be less irritating. policy-rc.d is just way too opaque, and it has the wrong affordance. I don't want to set "don't run daemons" as a system-wide policy; I want it as a setting for a specific call to apt-get. It's another example of a system being used because of what it can do, rather than for what it was designed for. This rarely ends up looking good.

atoponce4• 12 years ago

The post is missing a critical point: per the Debian policy, services should only bind to localhost upon installation. Further, Debain advertises itself as the "universal" operating system. This implies that the defaults are sane for many people, and if the administrator of the system installs a service, the administrator probably wants the service running. So, the only thing left to do, is change the bind address, and restart the daemon. Further, per the Debian policy, when a daemon package is installed, it's configured to start automatically on boot for run level 2. The policy is written in such a way, that Debian system administrators know _exactly_ the state of their system, after installing packages.

This post is written from the Red Hat way of doing things, in that you install the daemon, do configuration, then start the daemon. However, there is no consistency with Red Hat systems. Some package maintainers will have the daemon configured to start on boot, some not. So, after configuring and starting, you also need to use chkconfig(8) to check if the daemon you just installed is ready to start on boot. More often than not, it's "off", which means it won't start. Painful thing to discover when the server restarts, and customers are calling in that their website isn't loading, or they can't access their NFS share.

IMO, I have come to find the Debian way of installing services more sane, than the Red Hat way. I want my system working, and I want to know the state my system is in when making major changes. Red Hat is just too inconsistent here, and it doesn't help that they have the /etc/rc.d/init.d/ historical baggage.

fsniper2• 12 years ago

I really like the way it's now. But in some edge cases you need the service stopped in the first place.

For example RabbitMQ initiates a database which denies cluster configurations if intact. So in Debian way if you are using some configuration management tool to start a RabbitMQ cluster you have install/stop/resetdb/restart cycle which is prone to failing.

So some edge cases need this kind of 2cents.

atoponce4• 12 years ago

This is why upstream developers should use /etc/default/ to place daemon configuration files into. Then the configuration file could have "START=no", and even though the policy would start the daemon by default, the upstream developer could prevent this from happening.

CarlHoerberg• 12 years ago

easiest way i've found is: RUNLEVEL=1 apt-get install ...

kolev• 12 years ago

Nice! You need more upvotes! I think `policy-rc.d` is a hack.

Install Debian packages without starting daemons

Related Stories