Ask HN: Good multi-server deployment practices

https://news.ycombinator.com/item?id=1666239

by maverhick • 16 years ago

1 2 16 years ago

news.ycombinator.com

Hello, what are some of the better ways to deploy code/modifying databases across multiple machines? How do you guys do it and keep your processes swift, servers secure and have the ability to roll back if something breaks.

kls• 16 years ago

We script everything so that it is an automated script further all of our process is managed in JIRA. So when we do a release, the production manager decides whats in and what out via JIRA, based on that, our script takes those ticket numbers, goes to our source control system and moves the associated code into a release branch, tags it and automatically deploys it to a testing server, the release is verifies and tested. If good, the production manager then goes back to Jira and marks it as tested. The system then deploys the release to production and once verified merges that branch back to the head.

OK so what happens in the actual release? We do a rolling release in which the script goes to the router and re-routes traffic to the other servers (basically takes a server off line). The script then deploys to the server, brings it up verifies it and then turns traffic back onto the server.

A couple of principals that we live by are that are back end applications are REST services, we don't do Java, or RoR, or .NET apps, we do REST middleware with a language, ours is Java but it could be any. This is a huge distinction because it means that we don't do JSP, JSF or any of the other server side frameworks. This makes our server side very easy to mange.

Also we don't use session. All of our services are stateless. Which means we can drop in a new service or section of an overburdened service to dedicated hardware with out all the implications of session. This allows us to handle traffic without having to deal with the context of that traffic. If we experience heavy load on the profile service we can drop in 4 new boxes with the profile service on it and route request to those boxes. We don't have constraints that require them to go back to the same box they have a session on or complex session management infrastructure to ensure context is there for a user when a request comes into that box. Basically any request, at any time, can be services by any box, with no need for external information from outside the request. It is completely stateless.

On the database side all developers write delta scripts that take the database from A to B. We also take the cumulative patch set apply them together and generate a clean version x.x DB script that is more efficient. when a release goes out the scripts are run against a DB that is taken off line and that DB is brought up in unison with the app server that is at release x.x.

As well with the DB, we have a script that exports fairly static information that is in the DB out to static json files on the web server.

Finally, we don't template html files on the server, we write JavaScript UI apps that pull data from the json files and the REST services, we use a CMS to do templating and the CMS exports static HTML to the web server. Dynamic info is overlaid on the client side via JavaScript. This makes the web assets deployments drop dead simple.

maverhick• 16 years ago

Thanks a lot. Really helpful!

Ask HN: Good multi-server deployment practices

Related Stories