Debops (2014)

http://enricozini.org/2014/debian/debops/

by chei0aiV • 11 years ago

190 93 11 years ago

stephenr13• 11 years ago

I like this a lot, and I love the term DebOps.

The way developers somehow think DevOps is (or should be) an abbreviation of "Developers doing/replacing Operations" is terrifying to me.

I'm also in the same boat as the author, in that I recommend and target Debian Stable + Backports (and some vendor/community repos when required).

quanticle• 11 years ago

The problem with targeting Debian Stable + Backports is that your packages are going to perpetually out of date unless you go to the trouble of packaging up your dependencies as .debs and running your own apt server. Targeting Debian stable is fine if you can get away with using three year old libraries. But for a lot of languages, like Python, 3 year old libraries are at a significant feature disadvantage when compared to their current counterparts.

calpaterson• 11 years ago

Debian stable includes pip and virtualenv. If you want to manage your python dependencies for yourself that road is definitely open to you.

annnnd• 11 years ago

When you say "out of date", you probably mean feature wise, not security and stability wise? For me that is an acceptable tradeoff for most of the packages. Most of the stuff doesn't evolve at such pace that I should have the latest and greatest (I am talking about core libs and similar). And the packages where I miss some new functionality, I can just upgrade them manually. Take Firefox for instance - if you don't like/trust Iceweasel, you can just install FF to /opt/ and trust Mozilla for upgrades. For me this is win-win situation. You have a stable and secure foundation and you put bleeding edge stuff on it when you need it.

bbrazil7• 11 years ago

A bit over a year ago I researched what "Devops" means, and the answer seemed to be Developers being able to push code to production without having to involve Operations. This sounds like a good goal to have (presuming you've got good unittests etc.) as it removes an unnecessary friction for developers.

What it doesn't include is all the higher-value work the people on the Operational side tend to provide, like thinking about rollbacks, machine failure, network failure, provisioning, capacity planning, change/configuration management, security, monitoring etc - which is not to say that all developers don't think about these things, their focus tends to be on developing product rather than these non-functional requirements.

I'm presently in a world of Ubuntu LTS+a few manual backports for similar reasons as the authors. My home systems are ansible managed, giving me things like wireshark installed everywhere if I need it and new machines automatically hooked into Prometheus monitoring (which is much easier with my debs). I've seen what happens if you try to manually manage machines, and know that a small bit of upfront work will bring dividends later.

Writing the core code is just one critical step in running software long term, let's not forget the rest of the critical steps to keep it running in a sane way in the future.

chei0aiV9• 11 years ago

https://packages.debian.org/prometheus

stephenr13• 11 years ago

Many people have different interpretations of the term 'DevOps', but what you're describing in that first paragraph is Continuous Deployment.

Yes, it's related to (in that it's often a beneficial result of) but it's not explicitly the same as DevOps, by any definition I've seen, good or bad.

skarap3• 11 years ago

The way developers somehow think DevOps is (or should be) an abbreviation of "Developers doing/replacing Operations" is terrifying to me.

And what about the way management thinks that's when you don't have to pay Operations people (those guys who always try to stop progress in your company)?

stephenr13• 11 years ago

Sounds like another way of saying the same thing to me, no?

No one says "we don't need someone to setup/maintain our servers".

DevOps is, in every interpretation I've seen, about who/how it's done.

saurik• 11 years ago

I build my software targeting Debian Stable + Backports. At FOSDEM I noticed that some people consider it uncool. I was perplexed.

I am also perplexed... what are these other people doing?

Sphax3• 11 years ago

Deploy on CoreOS maybe.

chei0aiV9• 11 years ago

They are doing stuff like this:

https://news.ycombinator.com/item?id=9952356

chei0aiV9• 11 years ago

To the downvoters: the comments on that post mention a lot of deployment methods that people not doing Debops are using.

KaiserPro• 11 years ago

Yes, working smart not hard.

The value of going home on time each night, because there is a wealth of google searches to expose and workaround known bugs should never be underestimates.

Sphax3• 11 years ago

How do you not bundle in dependencies when coding in, say, Java ? The Java packages on Debian are for the most part outdated. It's way more practical to just use Maven, build a fat jar and be done with it.

chei0aiV9• 11 years ago

To not bundle dependencies, use the Java packages in Debian stable and if you find one with a missing feature or unfixed bug, then backport that from Debian unstable or from upstream.

bbrazil7• 11 years ago

Debs and Rpms are fine for configuring the machine, as soon as you have it where machines and application services aren't a 1:1 relationship that will lead you in the direction of dependency hell. As each application will want slightly different versions you'd have to be very careful which version you installed.

You can cheat a little. For example I am using some system debs for python packages, but if there were more than a handful of python utility packages that can get unmanageable fast. That leads naturally to the fat jar/virtualenv/static binary approach with manually managed dependencies which gives you isolation between applications, and all the associated costs.

Sphax3• 11 years ago

And when the library is not packaged in the repo, should I then build the package myself ?

If we were in an ideal world we would do that, but unfortunately we're not, and in my experience it's way easier for the dev AND for the sysadmin to use Maven and build a fat jar.

_yosefk• 11 years ago

How do you not bundle dependencies programming in anything? If you want to be portable across Linuxes, you can't even count on tcsh or env being installed at the same path. Counting on the stuff installed with the system unleashes a world of pain on the user.

icebraining3• 11 years ago

Yes, but many of us have no intention of being portable across Linuxes. Our software is designed to be deployed to machines we control. If users want to deploy it to some other distro, they're free to package it themselves.

kimballkinnison• 11 years ago

Targetting a given stable distribution means that the distribution is bundling all your dependencies.

If you're upset about that, learn how to instantiate containers on your chosen platform :-)

davexunit• 11 years ago

you can't even count on tcsh or env being installed at the same path

This is what tools like Autoconf and pkg-config take care of. Lots of people use them to discover executables and libraries and generate files with the right machine-specific variables in them. You should never assume that binaries are in /usr/bin or that libraries are in /usr/lib. A lot of packaging issues are caused by such assumptions.

If you use the right tools, you don't have to bundle your dependencies. Bundling introduces a serious maintenance burden onto the developers and the packagers. It's easy to avoid bundling for C libraries and things, but with the prevalence of language-specific package managers, it's become a harder problem because everyone just assumes that you will fetch dependencies through it and never use the system package manager. It's a sad state.

technomancy2• 11 years ago

It depends who your end users are. If the people responsible for running the code do it as their full time job, then yes, bundling the dependencies is more practical because you have a team of full-time engineers to handle security updates, rollbacks, etc. But if you're pushing it out to people who expect to just be able to fire and forget, not relying on your distro to manage dependencies for you is going to result in a world of hurt.

skarap3• 11 years ago

Never understood why debian ended as the distribution of choice of the DevOps-related movements. Ubuntu is understandable - developers used it on their desktops so when they orderer their first VPS, they chose ubuntu. But why debian?

Also why it looks like RHEL family is completely out of fashion? Is it considered too "enterprisy"?

juliangregorian• 11 years ago

Why Ubuntu? If you're not writing GUIs, Ubuntu is just a less-stable Debian.

jnbiche• 11 years ago

Why Ubuntu?

PPAs.

onli• 11 years ago

Also why it looks like RHEL family is completely out of fashion? Is it considered too "enterprisy"?

Well, look at the website: https://www.redhat.com/en/technologies/linux-platforms/enter.... That won't appeal to anyone outside of an enterprise chair.

bbrazil7• 11 years ago

My read of it (as someone in the Debian/Ubuntu world) is that the CentOS split meant new sysadmins and users tended more towards Ubuntu.

RHEL/CentOS is still around, but you don't hear about it too often.

sspiff• 11 years ago

Debian is fairly stable, has been around for ages but is still active, is completely free and community-driven, and most people are familiar with the package management and config file structure on Debian (or Debian derivatives).

On top of that, it was already a pretty popular distribution. The more people use it, the more documentation becomes available, and the more new people will pick it up.

vacri• 11 years ago

Ubuntu tries a lot of different things, and the environment swings around wildly. Sometimes you'll see a howto that has different steps for each of the past four or five releases (6mo apart). Debian is much more considered in it's changes, and is better suited for server environments. The Debian philosophy is quite clear, and you never have to keep an eye out for encroaching adware or phone-home stuff in the core systems. Debian also does less 'magic stuff' for you; for example, if you mistype a command, Debian will just 'not found' it, whereas Ubuntu will suggest-sell you a command... which takes time for the results to be parsed and presented.

On the desktop, Debian has quite a few more rough edges, and you wouldn't really recommend it for a newbie.

Regarding redhat being out of fashion, part of that is that Ubuntu was a big drawcard to bring devs to linux because they focused on polishing the desktop, so the debian family got an influx of users. Personally, I don't like redhat tooling as I find them full of gotchas and their output is usually full of chaff and hard to parse. That may be just a personal taste issue; I'm sure plenty of redhat admins find debian tooling weird and odd.

Kabacaru2• 11 years ago

So I develop PHP. The reason I don't use Debian stable is because the latest version of PHP on there is 5.4.41, which is behind the "old stable version". That means security fixes only.

No bug fixes that resolve problems, or modern functionality (that release is over a year old), and will be EOLd in 1 month. That means a big delta of change that will be needed to handled when debian finally does get round to upgrading. Large deltas of change mean lots of risk.

It's much better to stay further up the crest of the wave and handle more regular updates, to minimise the size of the risk I'm bringing into my code at each release, than it is to stick to an old version and not handle the stream of new functionality that's coming in as it arrives.

0x0• 11 years ago

Debian Stable has 5.6.9 at the moment, https://packages.debian.org/jessie/php5 - and shipped with 5.6.7 initially.

Security updates for php5 in Debian seem to have changed from backporting security fixes to staying up to date on any given upstream minor branch, including other fixes. https://security-tracker.debian.org/tracker/source-package/p...

Kabacaru2• 11 years ago

Gah, well there's that argument out the window. We tend to use docker on CoreOS here, so it's not much of an issue.

Maybe next time I stand up a real VM I'll look at using Debian stable then.

stephenr13• 11 years ago

For more up to date PHP packages on Debian, try http://dotdeb.org

gog4• 11 years ago

+1 for DotDeb. For Jessie I would use the stock packages.

zufallsheld• 11 years ago

There's also DebOps[0], a collection of ansible playbooks a " Debian-based data center in a box".

[0]https://github.com/debops/debops

nickjj• 11 years ago

I remember going over with the author on what we should name the domain[0]. I figured it was pretty unique.

[0]http://debops.org

olgeni7• 11 years ago

I recently had to develop an Erlang application targeting Debian "stable". It was humiliating at best. Countless hour were wasted trying to get around bugs in prehistoric packages, or building new (and working) packages from scratch.

Never again.

gog4• 11 years ago

Debian Jessie just came out 2 months ago and as far as I can tell the packages in it are pretty recent.

Were you targeting Wheezy?

olgeni7• 11 years ago

Yes - in the beginning.

If you look here https://github.com/erlang/otp/releases you'll see that - even on Jessie - 17.1 is unfortunately way behind.

logicchains• 11 years ago

Live dangerously: deploy on Arch. Then you'll never have to work with outdated anything again ;)

Actually, although I've been using Arch Linux for over two years now (not in production; I'm not brave enough), I've hardly had any issues at all with its rolling updates. The worst was just having to manually delete some Java binaries from /usr/bin when the way it handled Java was updated to allow Java 7 and 8 to exist side by side.

fensipens• 11 years ago

The thing with "rolling-releases" is that updates will bite you in the ass exactly then when you have absolutely no time to deal with them.

So even if the system boots, it doesn't mean anything because booting a system is only one out of 100+ use cases that you will never be able (or willing) to test after each pacman -Syu..

sleepydog• 11 years ago

I know a lot of people who use Arch. I myself used Arch for about 10 years. Everyone I talk to has glowing reviews of it, and they've never had any issues.

I stopped using Arch about 3 years ago after my system became unbootable after an update. It was no the first time. In the past, I would be fine reading update notes and fixing the issue. But since I started troubleshooting servers at work, I have zero patience for doing it at home.

One other annoyance that comes with rolling releases is that you should update more often, to avoid making bigger (sometimes conflicting) changes to your system. You end up reading release notes more often. I could turn on automatic updates, but I've been bitten by that in the past.

Arch also encouraged me to tinker, and I was much more likely to make breaking changes to my system than I am now just running Ubuntu. If I had the time/energy to try out new distros at home, I would probably try Nix or something similar, that emphasizes rollback capability.

olgeni7• 11 years ago

Arch has 18.0 - better indeed, but you need to keep an eye on GitHub to find out which is the actual latest version :|

mkhpalm2• 11 years ago

Did you try this? apt-get install -t wheezy-backports erlang

One wheezy, that would have put you at 17.1 in less than a minute.

olgeni7• 11 years ago

I'm actually developing on 18.0.2. At least 17.5.x would have been better.

The fact that erlang.org does not list minor releases at all probably doesn't help. One can go to erlang.org and think that 18.0 is the latest...

nvarsj• 11 years ago

Using Debian isn't a panacea. Neither is any particular distribution. Anecdotally, the only personal server I've had hacked was running Debian - due to it using an old openssh version (the issue was not present in later versions). It was fixed quickly, but I got 0dayed.

stephenr13• 11 years ago

The only server's I've seen (and subsequently replaced, with Debian) breached were old CentOS boxes.

The key thing here is not "Debian is bad" or "CentOS is bad" it's that you need to keep up to date with security patches. For Debian that usually means a combination of using the Security Apt repo, and for things like OpenSSH, using the Backports Apt repo.

I do agree that Debian isn't a silver bullet, but in my experience it's much easier to work with from a setup/management point of view than CentOS, particularly for small shops that are't heavily invested in a full-blown CM tool - shell scripts and/or Debian config packages [1] can be used to fully provision one server or fifty.

[1] http://debathena.mit.edu/config-packages/

HeXetic• 11 years ago

Am I the only one who finds this style of usually-just-one-sentence-per-paragraph almost completely unreadable on a visual level?

ecesena• 11 years ago

Please edit title with (2014)

throwaway0189• 11 years ago

Related to this: Outsourcing your webapp maintenance to Debian

https://feeding.cloud.geek.nz/posts/outsourcing-webapp-maint...

Debops (2014)

Related Stories