Ask HN: Hacking concerns: does it make a difference which OS I run my SAAS on?

https://news.ycombinator.com/item?id=9055206

by hoodoof • 11 years ago

38 33 11 years ago

news.ycombinator.com

So I'm launching a new SAAS startup.

Does it make any difference from a security perspective if I use Linux or FreeBSD or something else?

patio112• 11 years ago

Not much. Use an LTS version of a distribution which you understand well. You're more likely to have appsec problems caused by yourself rather than being compromised by problems in the OS.

Make sure you're able to do firedrills like e.g. upgrading the version of Nginx or OpenSSL which you use in response to a vulnerability in them getting discovered. We've been doing a lot of that these last two years, and it probably won't get any less common.

WestCoastJustin• 11 years ago

Going to echo patio11's comment. But add a little too. Do not run Wordpress / Wordpress Plugins, or Drupal, on the same system you are doing anything with user data. These continually have security issues and you will almost certainly be owned [1, 2]. Out source blog hosting to one of the many hosting providers that maintains the software install for you. The chances of actually getting owned running stock Apache, or Nginx, are extremely low. You really need to watch what third party apps you install, like Wordpress, Drupal, or an unsecured phpMyAdmin.

ps. Make sure you have source code and database backups happening on a regular basis. If the server dies tonight, are you able to restore it to a point in time? What about getting the application stack installed quickly? Think about these things.

[1] https://wordpress.org/news/category/security/

[2] https://www.drupal.org/security

cat9• 11 years ago

For any X, getting rooted is still often a matter of someone with 1000 botnet nodes to burn running a for loop against an IP range. At which point, your odds of getting rooted are the product of how desirable that target is to write for loops against and how long it has been since you last applied security updates.

The reason WordPress tends to be root city is that there are a great many installs out there, of which many have never had step one done to harden it, and were last given security updates ~ when they were installed.

I would still outsource WordPress, but that's mostly in the vein of "the cost of outsourcing is less than the cost of me having to think about it one hour a month, while 1 hr/mo is a reasonable floor for the time cost function, but the actual value will probably exceed that at least once in a given year."

A WPEngine subscription costs $30 to "I don't care, why are you wasting my time with numbers this small." Developer time costs between $75/hr and "everything is on fire and you can only put out one fire at a time." Ergo it makes economic sense to configure nginx / DNS once, then outsource further complications to paid external support.

collyw2• 11 years ago

I know that Wordpress has a bit of a bad reputation, but this is the first have heard that Drupal has similar problems.

Is there anywhere you can check how good or bad a particular CMS or framework is regarding security?

michaelbuckbee2• 11 years ago

The OP also asked about a difference between distros (DIY) and "something else" - and I do think that there is a large difference in the security + maintenance surface area that you have to contend with on a Platform As A Service like Heroku versus your own box.

What's more I think you hit on exactly the reasons why: I'm much more likely to screw something up by misconfiguration or lack of attention than a team of security and maintenance people at a company whose sole job is to make sure that I don't get burnt by these things.

OP is trying to get a SAAS off the ground and not (presumably) doing something with handling distro security and server maintenance - for the minimal cost difference in getting started it's hard to see how the time+effort difference in being your own sysadmin is worth it.

patio112• 11 years ago

FWIW: this is imminently sensible advice, too.

logn3• 11 years ago

Heroku can be expensive. You're looking at $240/yr just for SSL (vs. $0 - $30/yr for non-wildcard SSL). For a 1GB VPS, a dyno costs ~3x as much as Digital Ocean.

For protection from cyber crime, you might be better off paying an insurance company than Heroku prices.

stock_toaster• 11 years ago

In addition to what patio11 said with regard to firedrills for updates, make sure you use some type of repeatable automation. chef, salt, puppet, ansible -- just pick one and use it... _religiously_

yeukhon3• 11 years ago

I don't mean to offend you, but I guess offense well taken. If you are asking this question, it's really important you hire someone with really good system knowledge and security knowledge before you launch your SaaS production, besides having someone doing a proper security testing. If you are running on AWS then make sure you have a proper IAM policy and proper security group settings. If you are running applications then containers are probably a nice addition. Basic WAF is essential. Even enable fail2ban even if your servers are in private subnet. No password-based SSH. UPDATE UPDATE UPDATE. These will get you enough before someone finds a remote shell exploit take over your server or someone leaks secret key in github repo or before you get XSS pwned.

todd38344• 11 years ago

I disagree, unless your SaaS is storing highly sensitive data.

TallGuyShort• 11 years ago

I wouldn't use a SaaS if I wasn't convinced you considered everything about me highly sensitive.

yeukhon3• 11 years ago

Sorry, but if your server is hacked, you lose trust, even if all you do is keeping my five years old cousin's drawing. deb.haskell.org didn't store anything sensitive, but as a server storing the .deb it was hacked and has impact on possibly ten of thousands of users. Just an example.

meowface3• 11 years ago

If you're in the bootstrap phase and don't have a lot of employees, then yes, I agree with you.

But as soon as you can afford it and/or as soon as you get big enough, you should have at least one FTE who focuses purely on security. This applies to any company with an Internet presence.

todd38344• 11 years ago

I suggest you pick an LTS version of an OS that is extremely popular like Ubuntu. That way you will have lots of people paying attention and patching security flaws ASAP. Then make sure you update regularly/automatically. Put it behind a strong firewall, AWS makes this really easy but you can do everything you need to with iptables. Disable password based auth in ssh and force key auth.

I disagree with yeukhon. You don't need to hire someone just because you asked that question. Unless you are storing highly sensitive data, then I'll leave that up to you.

Just remember to constantly update your server.

philips• 11 years ago

It generally comes down to picking something you understand how to update and manage. Most of the base OS security is a practice in staying up to date and applying patches very regularly.

There are two types of Linux OSes in general: versioned release and rolling release, which I will explain below.

When choosing a versioned release distro you need to pick something that you feel comfortable you can update on a regular basis without breaking your application dependencies AND plan for the eventual migration to the next versioned release of the distro. Any release you choose will be supported for several years with security updates and then fazed out. When that end of life date hits you must move your application to that next version. Examples of these types of distros are CentOS, Ubuntu LTS, etc.

An alternative is building your application with containers and choosing a rolling update distro. The advantage for security is two fold:

1) It makes it easier to roll out new versions of your application on a regular basis because it is possible for two versions of your app to live side-by-side. And for a new web based project most of your security problems will likely be from your web frameworks, custom code, etc not from the underlying OS. So, having a system in place to get those updates out will be critical.

2) The Linux OS can be updated on a much tighter cadence because your application isn't relying on much from the OS besides the container runtime. Rolling release server distros include Arch Linux, CoreOS, Gentoo, etc. CoreOS even fully-automates the OS update process to remove the burden of remembering to audit and apply updates.

Using a rolling release distro without containers is an option too but can be bumpier since you will regularly be going through migrations like /usr/bin/python being 2.7 and then migrating to 3.0 without you noticing.

The final thing you might consider, particularly if operations aren't your forte, is using something like Google App Engine, Heroku, etc. These systems are ideal for building a prototype and focusing on the initial minimal viable product.

disclosure: I work at CoreOS

jmbwell• 11 years ago

What is your exit strategy? What will your buyer value? What is best for your product? What do your engineers understand?

nikanj• 11 years ago

Don't go for analysis paralysis

zhte415• 11 years ago

My 2p:

Use something you're familiar with, and that you have a good source of information flow, by which I mean, subscribe to security updates (including for whatever stack you put on your server), and certainly have a test server simply for playing with things, getting used to functionality.

I have a shell app on my phone, which I love, so I can log in at a moment's notice, wherever I am, just to check things out or apply a change.

The Linode and DO tutorials are pretty nice, and cover the vast amount of eventualities. If you're unsure which to choose, go with the one that has more documentation for your use case. DevOps is not cloak-and-dagger mysticism; having control and understanding of your stack is, imho, far more important than choosing between a distro to run it on. Just make sure your server is locked down and that you regularly apply updates (that you've tested on your test server).

PeterWhittaker• 11 years ago

If you are not using someone else's proven platform, then yours will be hacked, possibly several times, at least until you've learned how to secure it and have kept it secure for some time.

With that in mind, you must, must, must do a few things:

1. Get really, really good at rollback and recovery. Make sure you can completely rebuild your operation from a clean system in as short a time as possible, with no data loss. Certain types of compromise will require this, it will be the cleanest, safest way to recover.

2. #1 requires you to be really good at backups. Unless you have experience here, they're harder than you think.

3. Notice that #1 starts with "rollback": You will occasionally be your own worst enemy, at least until your release processes get really, really good. Every now and again you will apply something that breaks something else. Being able to get back to a known good state quickly and easily will save you countless hours and much stress.

4. Find an experienced penetration tester that you trust implicitly. Have them review your operational architecture; they will find flaws (that's just on paper). Have them review your hardware and software configurations; they will find flaws. Listen to their proposals for risk mitigation. Once you have a system that passes their paper attack, have them actually kick at it, from three different places: one, from the outside, from the Internet everyone comes from; two, from just inside, from within your first tier, between outside interfaces to the world and app logic; three, from deep within, behind the app tier (using test data, of course). The three reports should be very different, with increasing levels of access and vulnerability as they go (because you are letting them past the mitigations). This will give you considerable information about how at risk you are.

(I know quite a good one, does this sort of thing professionally for government, private sector, the military, police forces, etc. He'll break your stuff. And tell you how to make it better.)

5. #4 implies, strongly implies, that you have a restricted access test environment that matches your operational environment as closely as possible, with two exceptions: one, no production data; two, it may be a patch or two or a major update ahead of production, since it's where you will soak changes to make sure they work properly and break nothing.

6. Managing #5, #3, and #1 means you must, must, must understand and get good at change management. And risk assessment and risk management.

IT Security is mostly about governance and management. If you focus on technology first, you've already lost.

Ask HN: Hacking concerns: does it make a difference which OS I run my SAAS on?

Related Stories