Securing an Ubuntu Server

http://www.andrewault.net/2010/05/17/securing-an-ubuntu-server/

by chaosmachine • 15 years ago

168 52 15 years ago

zdw• 15 years ago

I tend to find that minimalist OS's, such as plain Debian installed with no frills or OpenBSD tend to be better candidates for security sensitive machines.

Some of the utilities mentioned like Tiger haven't been updated in years.

No mention of chroot/jails/zones/etc, which can go a long way to nullifying intrusions when they happen, or integrity tools built into the system (using package managers to check for changed binaries, or checksum tools like like tripwire/AIDE/radmind)

While it does have some good points, it also strikes me as "cargo cult system administration", specifically "Following a security recipe": http://blog.lastinfirstout.net/2009/11/cargo-cult-system-adm...

viraptor2• 15 years ago

Getting a minimal distro also gives you the possibility to install only the things which are needed, instead of uninstalling a lot of stuff which came in by default.

Additionally - if you are serious about monitoring, you should ship the logs to some external host in realtime, so they can't be deleted / changed.

Running services in jails is definitely a good idea. Additionally configuring Apparmor and Selinux could be a good idea too. Or even switching to a grsec kernel. For the consistency checking, I prefer Samhain.

krakensden• 15 years ago

I thought Ubuntu had AppArmor on and configured by default for many/most of the popular services (Apache etc). I often see 'updating AppArmor profile' log messages in system updates, in any case.

orthecreedence• 15 years ago

Agreed. If running a server, I'd choose Debian or Slackware over Ubuntu any day.

rtaycher• 15 years ago

Isn't Ubuntu server a fairly minimal install(not OpenBSD minimal but still).

Klinky• 15 years ago

The Ubuntu Minimal CD is one of the leanest distro install options available.

https://help.ubuntu.com/community/Installation/MinimalCD

rwmj• 15 years ago

That's just a small Ubuntu CD which downloads packages on demand (instead of keeping them on the CD).

If you really want a minimal distro, install FreeBSD or build a Debian root using debootstrap and just the daemons you need. I have a Debian machine (acting as a firewall) which is ridiculously parsimonious -- 'ps ax' fits on a single page and most of that is just kernel threads.

_b8r05• 15 years ago

In defence of the article, it has some good ideas. On the other hand it completely overlooks what people should be doing to secure their systems, regardless of OS.

People should look at the role of the system and the information assets contained therein, then determine (if possible) what types of attack and attacker might be able to get in, and how - that tells you what you need to do.

If you're not able to do the latter, but can do the former then you're still able to secure your system. A classic example would be a firewall. Your firewall contains no information assets beyond your ruleset. Your firewall however can be used to open up internal access to your network.

We can then say, "We believe our attackers will mainly come from the outside". How will they get in? We don't know. What can we do to stop them from accessing the firewall? Put a rule in to drop all traffic with a destination of the firewall's external IP address or network interface. Providing your routing's set up correctly, you've now put in an appropriate countermeasure.

Rinse, repeat for all of your Internet-facing assets, hosts, services and applications. It sounds a lot, but it gets quicker the more you do it. Most unlaunched startups with relatively uncomplex setups could do this in a day.

Here's another example. Our Apache server is accessible from the Internet.

Who can access it? Everyone.

Does everyone need to access it? Yes - we're using it to host our application.

Who is going to attack this? We don't know, could be someone skilled, could be someone not skilled.

How will they get in? An Apache exploit, an Apache misconfiguration, something further up the stack (like our app or PHP).

What do we do about it? We maintain Apache versions (to address the Apache exploit issue as best as we can).

We make sure all unnecessary Apache modules are disabled and that our Apache setup is as minimal as possible, then look at the directives on the Apache docs and change the settings to be as effective as possible (without breaking our app of course) - this addresses the misconfiguration issue. If we have to switch something on we now know the effect of that and if concerned can ask HN, monitor it or both.

As for something further up the stack, let's repeat the process on the way up until we're comfortable with what we have.

If you're writing down your reasoning and your results along the way this will help you get up to speed - it also means you can write your own hardening guide, or even integrate it into fabric scripts to save time in the future. You'll also massively increase your understanding of the software you're using in the process, so it's definitely worth doing.

bryanh2• 15 years ago

I go by the trifecta:

* Disable root login. * Change SSH port. * Install fail2ban or denyhosts.

fierarul• 15 years ago

Wow, I didn't know of fail2ban and denyhosts.

I actually have a home-made script similar to fail2ban.

bmelton• 15 years ago

Yeah, I've written a number of very long, very gruelling scripts that ultimately were all replaced by denyhosts.

On the flip side, denyhosts only works on sshd (to my knowledge) -- so the scripts I wrote to monitor tornado log files and block all the random attack vectors is still worth having around.

X-Istence• 15 years ago

I wrote a similar script way before fail2ban was ever created in around 2003 or so. When PF was first introduced. Since then I've just kept using it since it works particularly well for what I want it to do.

getsat3• 15 years ago

You're missing a really important ssh setting: disable password-based authentication!

However, this still won't protect you from vulnerabilities in your ssh daemon itself. Optimally, you would use Single Packet Authorization.

chanri2• 15 years ago

What if you can't disable root login (i.e. it's a cloud server)?

steve19• 15 years ago

Restrict root login from a particular IP address

AllowUsers root@112.113.114.115

I use AllowUsers to ensure only users I want can login from SSH.

Estragon• 15 years ago

Why can't you disable root login on a cloud server?

dedward3• 15 years ago

This is where firewalls come in handy...

Disabling remote root login isn't as big a deal as it used to be - as others have said, if someone gets your administrative account you're pretty screwed anyway - and if they get your password for sudo, it's the same thing.

Other than that, it really depends on what you mean by cloud.

rjbond3rd• 15 years ago

There's no reason you can't disable root logins on a cloud server. I do it every time.

SwellJoe5• 15 years ago

Actually, the trifecta would be:

1. Update your software.

2. Use strong passwords, or public keys, for all accounts, regardless of how unimportant they seem. A strong password is one with letters, numbers, and possibly special characters, 8 or more characters in length, and is changed periodically.

3. Don't run unnecessary services or services with a poor recent security history.

Disabling the root login is a legit choice, though if someone figures out your sudo-capable account, you're equally screwed, so use strong passwords always.

Changing the ssh port only stops the people who aren't targeting you specifically, and even those people aren't always stopped. It is trivial to scan a host to find where ssh is running, regardless of the port you put it on.

fail2ban or denyhosts are OK for preventing brute force attacks, but OpenSSH and most security sensitive software already has timeouts, among other measures for preventing brute force attacks. Which makes fail2ban more of a nuisance-prevention measure (i.e. you see fewer brute force attacks show up in your log reports, and thus making it easier to see the more dangerous threats), rather than a core security measure.

If you performed your three steps, but ran an insecure version of any service (particularly those that have elevated privileges), you would still be easily exploited. fail2ban or denyhosts would prevent most services from being exploited due to weak passwords, so that's probably not a hole you'd have. And, changing the ssh port does nothing for you if your system hasn't been updated and is running exploitable services.

I used to do security cleanup and post-mortems and data recovery as a contractor for folks who'd had systems exploited. 90% of them were due to running old versions of software, and the rest were due to weak passwords (which may or may not have been prevented by fail2ban; in a couple of cases the exploiter used the webmail client to brute force the weak passwords...most people don't include the webmail client logs in their security considerations, and thus wouldn't configure fail2ban to ban attempts on the webmail client).

I'm not saying your advice is not useful, just that it's something you do after you get your core processes in order.

Edit: Formatting.

chanri2• 15 years ago

Great advice! You sound like you have a lot of experience in server security.

What is your recommendation on strong passwords? Also, should all passwords on a system be different (login passwords for different servers, email passwords, etc...), and if so, how do you keep track of all of them?

Thanks

bryanh2• 15 years ago

Spot on advice, and a valid rebuttal. My trifecta was more in the spirit of securing against hundreds of brute force attempts, but I can't really knock your excellent advice.

_b8r05• 15 years ago

That's a terrible trifecta.

* Disable root login

If you have root login enabled by default, you're using the wrong distribution. This is 2011, not 1996.

* Change the SSH port

Why are you doing this? What will you achieve by it? In what way does this reduce your overall risk of compromise, and by whom?

* Install fail2ban or denyhosts

That's the only thing out of your trifecta that should be in there. Aside from that, here's why your trifecta isn't a good move:

You're still using password authentication for SSH. Switch to public key authentication. This, combined with fail2ban or denyhosts means bots are not getting in. Period.

You aren't addressing your attack surface area - look at what information assets are exposed to the wider world and how. Once you know what your information assets are and how they can be accessed (either legitimately or unscrupulously) from the outside world you know what you have to protect and how people can get to it. That way you can focus on real countermeasures.

What about your applications? Have you checked your code to make sure you're validating input correctly? How are you protecting against SQL injection or Cross-Site Scripting? How are you handling state?

I put to you an alternative trifecta:

* Know your assets

* Know your threats

* Use appropriate measures to protect your assets from your threats.

rm-rf• 15 years ago

"* Change the SSH port"

"Why are you doing this? What will you achieve by it?"

Changing the port does not improve security. It does, however:

- dramatically reduce the noise associated with the fleet of password guessing bots that hit open SSH server daily.

- make it reasonable to assume that a password guess attempt is specifically targeting your serve, and therefor consideration for escalation and follow up.

Signal to noise ratio. Less noise make it possible to discover the signal.

lamnk2• 15 years ago

For a firewall (frontend for iptables) I recommend csf: http://www.configserver.com/cp/csf.html

Features: stateful inspection, protection from different types of attack, ability to ban port scanning attempts, ban brute force logins for various services (ssh, ftp, ...), numerous configuration options but very easy to configure with excellent inline documentation. For more info see the link above.

SwellJoe5• 15 years ago

iptables already has all those features. I find "frontend" scripts like CSF confusing and distracting, because they throw in everything under the sun, including stuff that might make no sense in my particularly use case.

lamnk2• 15 years ago

Yeah, if you know the TCP/IP networking stack and iptables well enough. For most people, they do not. That's why sysadmins exist ;-)

Otherwise, can iptables ban those who failed more than 10 SSH logins?

zokier• 15 years ago

nmapping localhost doesn't sound very effective.

getsat3• 15 years ago

netstat -tap | grep LISTEN

beala• 15 years ago

Or you could just use the 'listen' flag ;)

sudo netstat -tpl

sucuri2• 15 years ago

I don't get why people recommend chkrootkit + failban/denyhost.

I would recommend OSSEC to anyone looking for a serious host-based IDS (it does all those tools do + a lot more and very light weight).

*link: http://www.ossec.net

dedward3• 15 years ago

I found that, while it looked great, for some reason on every install I tried (granted this was a couple of years ago, but I tried on various distributions and hardware) - the mailing engine tended to silently fail - which was fairly critical.

Denyhost, while not as sophisticated by a longshot, is dead easy to install and take care of the SSH brute force issue.

muppetman• 15 years ago

You should also look at the grsecurity kernel patch:

http://grsecurity.net

It can really help to harden a system by making it much, much harder to exploit buffer overflows, making it impossible for users to see any processes other than their own etc. It's a great kernel patch and I highly recommend it.

dedward3• 15 years ago

Securing a server is relatively easy - what people need to look past is securing the system as a whole.

Some day something in the servers or network you are responsible for will probably be breached in some fashion. You won't expect it. You'll find out. You'll panic.

Think past securing the perimeter and think about what happens when that fails... because it may. Are you using SSH keys internally to walk around systems? Are they secure? What happens if someone accidentally leaves a test account open somewhere and somehow, through dumb luck, ends up with root access to your kickstart server or whatever config management stuff you have?

Now - what will you do when you have to start from scratch? Everyone likes to say once a system is compromised you have to format - but that becomes a daunting task if you're talking about hundreds of systems including the systems used to manage those systems. Do you nuke everything from orbit? Can your business afford the downtime?

You need recovery procedures as much as you need security... and you need auditing procedures to ensure that your security and recovery procedures remain valid.... and you need auditing procedures to ensure those auditing procedures are followed.

Complexity also breeds problems... keeping systems simple also makes them easier to manage.

moe3• 15 years ago

Articles like this are amusing but also somewhat disturbing.

Ubuntu is based on debian unstable. It was created to escape the slow debian upgrade cycles and get more recent packages on the desktop.

So far, so good. But why are we now baking a server distro out of a desktop distro that is based on the unstable branch of a server distro? And who would put that on a server and try to "secure" it?

Tichy• 15 years ago

Maybe debians philosophy and approach is not the only valid one for security. Or people want to choose other trade-offs than the debian maintainers.

DarkShikari• 15 years ago

stable means "doesn't change very often". It does not mean "less buggy".

In my experience, Debian "stable" packages tend to be literally covered in bugs and security holes due to being years old. Only a few of the most popular packages get fixes backported.

moe3• 15 years ago

Debian "stable" packages tend to be literally covered in bugs and security holes

Would you mind backing this up?

Only a few of the most popular packages get fixes backported.

Debian must have many popular packages then, considering lenny has accumulated security backports for 1529 packages in amd64 alone.

They take security pretty[1] seriously[2] and considering it's all done by volunteers it doesn't seem nice to spread FUD about their work without any data to verify your claims.

  [1] http://www.debian.org/security/
  [2] http://security-tracker.debian.org/tracker

asdfor• 15 years ago

The first step on securing an Ubuntu server is switching to Debian (this sounds a bit trollish but oh well)

Securing an Ubuntu Server

Related Stories