Ask HN: Just replace my DBA: painless db admin and scale without the cloud?

https://news.ycombinator.com/item?id=498647

by jacobscott • 17 years ago

7 14 17 years ago

news.ycombinator.com

The recent launch of FathomDB is the latest of many articles on HN about databases, scalability, and the cloud.

What is the state of the art -- in terms of scalability, management, monitoring -- for people who need to keep their data in house? Maybe X where X:FathomDB as Xobni:Gmail? Or are we stuck hiring a DBA?

jacobscott3• 17 years ago

If it helps, I asked a more concrete version of this question focusing on PostgreSQL on stackoverflow: http://stackoverflow.com/questions/556325/postgresql-managem...

charlesju2• 17 years ago

Managed hosting works for us (Engine Yard), it is significantly cheaper than hiring your own DBA.

xenoterracide4• 17 years ago

having managed hosting doesn't mean you don't need to a dba. In my experience all the host does (worked for a managed hosting company) is set up the servers and keep them running, their help really can't help you with tweaking queries or tuning the db. I can't speak for on engine yard specifically. But I wouldn't bank on your host knowing anything beyond the basics of databases.

charlesju2• 17 years ago

Engine Yard has several excellent DBAs on staff. They tell you what queries are slow and tune the db to optimize for your type of traffic. That's why they are worth the 30% premium on top of EC2.

xenoterracide4• 17 years ago

There is no easy out, if you really need to scale, regardless of infrastructure you are going to need someone who understands databases. If you're lucky you might be able to find someone who can do more than one job. e.g. dba+sys admin. Just make sure that they really do know both things.

thwarted2• 17 years ago

Yeah, this kind of question seems really close to "I'm a business guy, all I need is a tech guy to implement my idea", which implies there's some way to avoid having to know how to do something that is core to the business.

jacobscott3• 17 years ago

I want to make it very clear that I'm not a business guy.

Think of it in the context of building a database-backed enterprise app that customers run in house. You don't want to ship a DBA to every customer...

jacobscott3• 17 years ago

It seems like I asked the question in the wrong context. Sorry about that. Given that scale is hard, how about monitoring, or management?

DenisM• 17 years ago

Try a commerical database, such as Microsoft SQL Server (FD: I work there as a developer). To bring up a recent example, it can build indexes without locking the whole table so you wouldn't need to jump throught the hoops like the FriendFeed guys just did with MySQL. Unless your time is free it will cost you less to buy SQL Server than to write your own "online index thingie" on top of MySQL.

And if your time is free, then you are probably a startup and you can get SQL Server and bunch of other MS tools for free via http://www.microsoft.com/BizSpark/

MySpace runs on SQL Server: http://www.sqlservercentral.com/blogs/steve_jones/archive/20... and if it's good enough for them it will likely be good enough for your site as well. I'm not sure if the scale-out story is any good (there are "partitioned views" which are basically "sharding" but I never used those), but scale-up story is as good as they come - it can efficiently utilize 64-cores of the HP superdome.

I don't know much about Oracle and IBM DB2, but I know that the latter has a free version without too many restrictions on it, so give them a spin as well. If ease of use is importnat to you, common wisdom has it that Microsoft's is the easiest to use among the top commercial vendors. NB: I did not verify the common wisdom, YMMV :-)

moe• 17 years ago

You're suggesting to use a commercial database but don't provide any reasons beyond what free databases can do.

PostgreSQL is pretty much on par in terms of features and performance with SQLServer and Oracle. So if you're looking for a free solution to roll your own then look no further.

The only reason I can see for going with a commercial DB is when you not only want commercial support (you can buy that for Postgres, too) but a shrinkwrapped "don't bother me"- solution with SLAs etc. Be prepared to pay through your nose for that, though.

In terms of usability there is no big difference between any of the commercial or non-commercial offerings. They all have their special peculiarities (which is why you always hire a DBA who is familar with your RDBMS, not just with "any" RDBMS) and the shiny GUIs that some of them provide are not normally of interest in day-to-day work.

iamelgringo• 17 years ago

The state of the art _is_ hiring a DBA. They get paid to be up on all the database monitoring, scaling, managing and management tools. This is why DB consultants charge $200+ an hour. It's pretty arcane knowledge, and for a business of any size, the database quickly becomes the lifeblood of the organization.

This is also the reason that databases like Couch DB or Amazon Simple DB or Google's App Engine are so appealing. They all have the promise of reducing db management headaches, at the cost of sacrificing features.

Ask HN: Just replace my DBA: painless db admin and scale without the cloud?

Related Stories