9 million hits/day with 120 megs RAM

http://tumbledry.org/2011/08/31/9_million_hits_day_with_120

by verisimilitude • 15 years ago

303 126 15 years ago

silverbax883• 15 years ago

I agree that this should be the standard, not an exception.

g-garron4• 15 years ago

I've been once on /., two times on Digg and all times my server crashed. I've improved it a lot since then, using different techniques. But never been hit by one of those again :(

New self hosted blogs can't get the load of peak times, because new bloggers start a WordPress blog, install on it a lot of plugins and never think about performance. I think most of us have made that mistake some time.

jellicle• 15 years ago

Sure. But it's not a mistake; it's a correct choice given current expectations.

What I don't understand is why the various CMS's don't offer automatic on-the-fly reconfiguration. It's 2011. We should be able to have the best of both worlds:

--when load is light, your blog software hits the database 42 times with every page loaded, no problem

-- when site load shifts from 1 page per hour to ten pages per second, the CMS should automatically, with no user intervention, say "oh shit" (perhaps audibly in the server room) and then automatically generate a static front page, and static article pages for whatever pages are being linked to, turn off the bells, turn off the whistles, email and tweet at the server administrator, etc. The CMS should cheerfully weather a storm all by itself. And when the load dies down, the static page should revert to a dynamic, 42-queries-on-the-database page, again without any intervention from the server administrator.

Does this exist anywhere, out of the box?

mike-cardwell• 15 years ago

I was slashdotted at the beginning of the year. Spent quite a bit of time at #1 for https://grepular.com/Abusing_HTTP_Status_Codes_to_Expose_Pri...

I managed to have it up and running again in about half an hour by converting some content which didn't need to be dynamically generated for every request, into static content.

Once things calmed down, I spent some time optimising it further and making lots of content static, which didn't need to be dynamically generated for every request.

I'm glad I did, because I hit the #2 spot on Slashdot for most of a day this very weekend because of this article:

https://grepular.com/Protecting_a_Laptop_from_Simple_and_Sop...

My lowly Linode VPS didn't even break a sweat this time.

senthilnayagam• 15 years ago

discovered blitz.io will keep me and my servers busy this weekend

tudorizer• 15 years ago

Kinda expensive :(

jbhelms• 15 years ago

I use http://loadimpact.com/

The free version can give you an idea of how your server will handle a small spike, and the other versions are per day so if you only need to test once or so a week as you finish a sprint then it wouldn't be a horrible cost.

senko• 15 years ago

I had a weekend hack project like blitz/loadimpact, was thinking of charging by the test (ie you pay what you use, which makes more sense than charging per calendar time).

Plan was to offer quick tests (ala loadimpact), whole-site test (you give it the url to start, and it'd hit the linked pages with some probability as well) and custom scenarios (a list of urls to hit in order, rinse, repeat), and API to trigger tests automatically (for integration in routine integration/regression testing).

Got the backend working, never finished the frontend/UI. If anyone reading this is interested, let me know in reply - might put up a quick working demo page for it.

kilburn• 15 years ago

Haha, looks like they could put some of the article's suggestions to good use there. Home page of blitz.io throws a 500 Internal Server Error right now...

fatalerrorx3• 15 years ago

same error ..was going to check it out to see how many server handles load

j_col3• 15 years ago

Very impressive, it's amazing how the "LAMP" stack continues to evolve.

Joakal2• 15 years ago

Looks more like a LNMP stack.

j_col3• 15 years ago

I'm using PHP-FPM myself and it is fantastic, major increase in performance when compared to mod_php. I still have not let go of Apache2 yet, but nginx is calling to me due to it's non-blocking design...

yatendra• 15 years ago

SNMP actually.

jacques_chester9• 15 years ago

The key points:

    1. Use caching.
    2. Use Nginx.
    3. Use PHP-FPM.

ashconnor2• 15 years ago

Why use FPM over Fast-CGI? I know Rasmus uses FPM and I would like to know the benefits.

jacques_chester9• 15 years ago

FPM stands for "FastCGI Process Manager".

I hope that answers your question.

easp2• 15 years ago

I think people are too quick to throw out Apache + mod_php. It remains the defacto deployment environment for most open source PHP apps, which means that it is the best understood and best supported. For example, if you deploy using Apache, you don't have to worry about converting .htaccess rules that wordpress and some caching plugins create into an Nginx equivalent.

People bitch a lot about the memory consumption of apache, but it's often overstated. They add up the resident set for each individual apache process and come up with a huge number, missing the fact that a significant amount is actually shared between processes.

Apache's memory consumption problems have more to do with how it handles clients. Each client ties up a worker process for the entire duration of a request, or beyond if keepalives are enabled (as they should be). That means that the memory overhead of the PHP interpreter is locked up doing nothing after the page is generated while it's being fed to the client. Even worse, that overhead is incurred while transmitting a static file.

I use nginx as a reverse proxy to Apache. It lets me deploy PHP apps easily and efficiently. Apache returns the dynamic and static requests as quickly as it can and moves on to the next request. Nginx buffers the result from apache and efficiently manages returning the data to all the clients. I used to have Nginx serve the static files directly, which is more efficient, but added complexity to my config. I chose simplicity.

A PHP opcode cache, like APC, is also a big win, because it cuts the overhead of parsing and loading the PHP source files. I'm not convinced of the value of other caching for most uses. CPU time usually isn't the scarce resource, RAM is. The DB and filesystem cache are already trying to keep needed data in RAM. Adding more caching layers generally means more copies of the same data in different forms, which means less of the underlying data fits in RAM.

snorkel• 15 years ago

So true. Apache + mod_php is very reasonable choice. Anyone who advocates using FastCGI instead of mod_php to "save memory" just doesn't understand what the actual memory footprint of Apache + mod_php really is, and how adjust the number of Apache processes.

In fact FastCGI still ties up an Apache process for the duration of the request: Apache hands the PHP request off to a FastCGI worker, then waits for that PHP worker to send back the output, so the Apache process is still blocked waiting on PHP in either scenario.

Also the overhead of Apache serving static content is miniscule compared to the amount of work a PHP does per dynamic request, unless the static content is very large, like large media files.

jacques_chester9• 15 years ago

> Nginx buffers the result from apache and efficiently manages returning the data to all the clients. I used to have Nginx serve the static files directly, which is more efficient, but added complexity to my config. I chose simplicity.

Funny, I felt the same way about ditching Apache entirely. Just one more moving part I don't need.

waffle_ss2• 15 years ago

Wonder how much faster it would be if PHP was taken out of the mix (looks like he's just just serving static pages anyway).

j_col3• 15 years ago

But he is using the APC cache, which is part of PHP, so I'm not sure he could pull that off? PHP-FPM is the fastest way he could be using that interpreter anyhow, as it maintains a persistent bank of PHP instances in-memory, with nginx acting as a proxy to it via a Unix socket.

icebraining2• 15 years ago

You wouldn't need APC if you weren't running PHP...

AlexC04• 15 years ago

What would you replace PHP with? Are you suggesting static files ?

g-garron4• 15 years ago

You are right, it would need less hardware for the same work. Two days ago I wrote this:

http://garron.me/linux/perfect-blogging-platform.html

But the way his comments work, they way he has implemented the infinite scroll it's great and I'm no expert, but I think that is not possible with static pages.

waffle_ss2• 15 years ago

Yes, obviously I meant just serving static pages. You don't need to run every page through PHP in order to have comments, dude. (I didn't mean completely remove PHP, just not make everything run through it)

dan_manges2• 15 years ago

Some interesting techniques in here (e.g. Faking Dynamic Features Using Inline Caching), but otherwise it seems easy to scale to this level when the majority of page content can be cached.

dan_manges2• 15 years ago

It's not apparent if the 9 million+ daily hits number is taking into account that peak hours will be higher than off hours. It would take 100 reqs/sec if the traffic is even throughout the day, but 375 reqs/sec if 15% of the day's traffic is in the peak hour.

onemoreact• 15 years ago

Yea, I see several diffrent numbers being tossed about but:

This caching (I’m using APC right now) got my page load times down to about 170 microseconds for most pages, and 400 microseconds

400 microseconds ~= 2,500hits/sec; 170 microseconds ~= 5,880hits/sec; Both of which seem reasonable for a simple PHP site using a single core CPU @ 1+Ghz.

mfjordvald• 15 years ago

Cross posting this from a comment I made on reddit: This is something I've actually worked extensively on solving and it's not quite as easy as this article claims it to be. In fact, there are quite a few too many draw backs to this method to any site that isn't largely static or updated very rarely.

* Whenever anything is updated the entire cache is invalidated and each item needs to be fetched again. This means you'll have some page loads being slow and others being fast. If you have a very dynamic website you will hardly ever even see a cached version.

* You can't cache things forever, primarily because when anything is updated the entire version namespace is invalidated. This means that if you have a site that isn't updated at all in a long time then the cache is still invalidated by the TTL and has to be updated. Of course, if you decide to cache forever and the version namespace is incremented then...

* You never know when your cache is full. Since the method of updating the cache isn't to invalidate keys but rather to just fill it with new keys, you will have a lot of stale data. This data will eventually have to get evicted from the cache. This means you don't reliably know when you need to upgrade your cache memory.

All that said. Version namespacing your cache is better than not caching at all and it's usually also better than having a lot of stale data as active keys. If you want to do proper cache invalidation in case you have a highly dynamic site then it's still possible, but it requires a lot more work, there's a reason for this famous quote: http://martinfowler.com/bliki/TwoHardThings.html

luigionline• 15 years ago

how about just using a CDN service. There is no need to play around anymore.

antihero• 15 years ago

That's about 100requests/sec, which isn't particularly amazing.

joshmlewis• 15 years ago

I would flip shit to get that many views. I'm sure I'm not the only one.

pestaa2• 15 years ago

On average that means 1.2 Mb memory available/request, which isn't particularly boring either.

michael_dorfman• 15 years ago

I guess my age is catching up with me-- my gut reaction on seeing the headline was: 120MB? That's a lot of RAM-- who has that? Oh, wait...

Luyt• 15 years ago

Indeed! I read it as:

9 million hits/day with 120 gigs RAM

Especially because of the scalability discussion around it. Only later in the article it dawned on me that he is using a cheap cloud instance.

harrisreynolds• 15 years ago

LOL... me too... I was thinking... who couldn't hit 9M hits a day with 120 gigs of RAM... just load everything in memory and walk away! 120 MB is certainly more impressive.

api• 15 years ago

This underscores how ridiculously overspecced modern servers are due to the bloat of a lot of modern software.

jacques_chester9• 15 years ago

Virtual machines change the economics of hosting for the host. A given VM has a lower administrative overhead per customer than shared hosting does. This will eventually drive out shared hosting, IMO -- except for single-application purposes.

lsc4• 15 years ago

"for the host" is key, really. the customer has to do a lot more work on a VPS than on a shared host, so I think shared hosting isn't going away.

Now, I agree with you that the shared hosting market will continue to move more towards constraining the user to a particular language within that shared hosting environment, as that allows the host to provide better service, but this has been true for a while. For the last 10+ years, if a shared-hosting provider allowed you to run an arbitrary binary, it only allowed you to do so as a second-class citizen, behind several layers of setuid wrappers and other (usually slow) security. Back in the day, if you wanted to run at reasonable speed on a shared host, you'd write in php, and use the supplied MySQL database, which conceptually isn't that different from what many of the language specific cloud providers do today.

The interesting thing is that this means shared hosting, usually with a well-supported but language constrained environment, is actually becoming the premium hosting option. VPSs, formerly the high-end option, are now the cheap stuff. Which makes sense; as a provider, sure, the VPS costs me more ram, but I can get an 8GiB reg. ecc ddr3 ram module for under a hundred bucks now. Ram is cheap. what costs me money is the support, and it is a lot easier to rent out unsupported VPSs than it is to rent out unsupported shared hosting.

If anything, with this new 'premium' image and advances in billing customs, I think we are seen a renascence in 'premium' shared hosting services.

noelwelsh• 15 years ago

I'd like to know how much the Joyent Smartmachine contributed to this. They make some bold claims on their website, and really do seem like a great alternative to EC2 (disk IO that doesn't suck!) if they deliver. Anyone have any experience?

winsbe01• 15 years ago

love it. gives me faith that the archaic machines i have serving can still hold their own!

krmmalik• 15 years ago

Quick Question. Would using Nginx as a front-end to Node improve performance in the same way it has done for serving PHP?

jjm• 15 years ago

Putting any proxy cache in front of your app will help. You can 'simulate' static caching of dynamic content. You will have to take a look at your headers however as most caches obey upstream caching headers to spec tightly.

jroes3• 15 years ago

Blogs really don't need a PHP, Rails, or anything backend. It's static content.

Here's how I think blogging should work:

1. Visit a web app where you create your blog post, add pictures, use your rich text editor, that sort of thing.

2. Click the "Publish" button, which generates static HTML and runs any other processing like tag generation or pngcrush.

3. Your static HTML gets pushed out to some server that does the hosting for you. It could even be one of those really cheap shared hosting providers.

If you really want comments, let someone like Disqus or Intense Debate handle it. Pretty much any dynamic feature you need can be outsourced.

jacques_chester9• 15 years ago

That's pretty much how MovableType used to work.

Fun fact: it was terrible. Most bloggers I know hated it.

Wordpress is a bucket of poop too, but it has the quality that when you press 'publish', the site is updated in seconds, not minutes.

(I believe MT has changed since then).

pwaring5• 15 years ago

It's how Greymatter and a few other systems used to work as well. The main problem was determining which pages should be updated, which usually resulted in the entire blog being rebuilt. This took several minutes if you had a few years of archives, and if anything went wrong part-way through the process you had to start it all again.

flyosity• 15 years ago

I've been using MT since 2003 and never thought it was terrible. Publishing times (full-site publishing) was always slow but publishing an entry only took a few seconds. The new Melody project forked MT and updated the UI, it looks great now.

burgerbrain• 15 years ago

"the site is updated in seconds, not minutes."

That is an implementation issue. That problem is in no way inherent to static content.

voltagex• 15 years ago

The problem with saying "Wordpress is poop" - as a developer, that problem report has no info I can actually use to fix anything.

mkr-hn5• 15 years ago

This is what you get with WordPress and W3 Total Cache (the caching plugin I use). It generates HTML files, then uses some fancy .htaccess magic to send requests to them. It can even push to a CDN. My blog survived being whacked by 3000 hits from HN over the course of 2 hours without blowing up.

Pre-gzipped pages are nice too. :)

tombot3• 15 years ago

W3 Total Cache really is a first rate plugin, we've had great success throwing Varnish https://www.varnish-cache.org/ infront of WordPress

EponymousCoward• 15 years ago

3000 hits over 2 hours is < 0.5 hits/second and nothing to write home about, sorry.

drzaiusapelord• 15 years ago

I do this with WP's caching plugin and a caching plugin for Drupal. It bugs me that these aren't part of the official distribution and need to be plugins.

rdouble• 15 years ago

Blogger used to work this way. You used to be able to download an archive of your site and host it on your own server! The thing is, unless your site is very simple and small, regenerating static HTML becomes more of a headache than maintaining a system that serves the content out of a database.

chopsueyar• 15 years ago

That is how viaweb worked. All product pages were static, and once you finished editing, you would 'publish' and it would create the static files for your online store.

whackedspinach• 15 years ago

I used Jekyll to generate my static content a few months ago/ Just type a text file, and then push it out to the server. I really thought that was cool.

sehugg• 15 years ago

Vignette StoryServer (http://philip.greenspun.com/wtr/vignette-old) .. if you see commas in a .html URL, thank them for it. And TCL as a scripting language, whoo-hoo :)

phsr• 15 years ago

In the realm of CMSs, RedDot works this way. It is a bake CMS, when you publish, it creates the whole site as a set of files. You can use it to generate php, asp, jsp etc pages, but it takes the data stored in a database and creates the files from the taxonomy stored there.

Maci• 15 years ago

You must be talking about Stacey then as it does some of the described "magic". :)

http://staceyapp.com/

handzhiev• 15 years ago

"If you really want comments, let someone like Disqus or Intense Debate handle it. Pretty much any dynamic feature you need can be outsourced."

If you are fine with your comments residing somewhere else, yes. I usually prefer to have them with me and to have them search engine indexable (that's free content created from users for me)

jroes3• 15 years ago

Good point. An unfortunate casualty. It might be possible to augment the statically generated blog posts with data via the comment providers' APIs periodically.

zachrose• 15 years ago

staticmatic deploy amazon_s3

FTW

wingo• 15 years ago

Very nicely done. Using JS to give personalized experiences seems to be the way to go. I suppose you could generate a JSON list of id,date pairs to reduce page bloat, if it matters.

[Edit: This is 100 qps. It's a lot for a blog, but is not an unreasonable load by any means.]

RyanKearney• 15 years ago

I'm not too sure I'd go with a load testing company that can't even keep their own website up.

http://blitz.io/

> Internal Server Error

mkr-hn5• 15 years ago

It's always exactly the time when you need your service to be up the most that it goes down. What matters is how they respond to it.

kowsik4• 15 years ago

thanks for the vote of confidence, will write up what happened soon.

gtklocker• 15 years ago

He could just say "tl;dr\n<machine specs>\n<I use static pages>".

ez772• 15 years ago

Naive question: suppose that you serve a low-throughput site, say with a total of 3MB of data (probably text files). What's the simplest way to ensure that those 3MB of content (very little compared to 120MB) live always in RAM? By this I mean not giving the server a choice =).

sbierwagen• 15 years ago

https://secure.wikimedia.org/wikipedia/en/wiki/RAM_disk

amalcon• 15 years ago

Point the server at a ramdisk, and set up a cron job to sync it from the regular disk?

kahawe• 15 years ago

I have to say I haven't tried that myself nor have I looked at the prices so my question is: All the fun of having your own server aside, why wouldn't I rather just run a site like that on something like amazon ec2 and stop worrying about hits and load even if it is just a personal blog?

schiptsov• 15 years ago

There was some post about a happy Win/IIS/CF guy - he definitely should read this.. ^_^

g-garron4• 15 years ago

As a lot of you have said: Static content is the key to success. You can name it:

- Movable type

- Drupal + boost

- Wordpress + SuperCache

- Jekyll or other static website generators

Better if Nginx is serving those static files, LAMP can be behind creating the static files.

I used that way with Drupal+boost for a lot of time and worked.

njharman• 15 years ago

hits/day and megs ram are orthogonal.

TylerE• 15 years ago

One thing to point out, based on my experience, is that you need about 10x or more peak throughput to handle a given average throughput. Spikes kill you.

michael_h• 15 years ago

By inlining the comments, he's reducing cpu time by...transferring extra data across the network?

todsul• 15 years ago

The difference between Apache and Nginx is that out of the box, Nginx is built for speed. Both are capable of thousands of requests per second, but Nginx arguably does it better with its event-based architecture (opposed to Apache being process based). The config syntax is also refreshingly simple, so converting .htaccess rules couldn't be easier.

We were recently paying a small fortune for hosting one of our websites. It was bumping up against memory limits even after a serious code rework and aggressive caching. Instead of upgrading we decided to test a new config using Nginx.

Now we run three sites, one fairly popular, on a 512Mb Linode with Nginx, APC, FPM, Varnish and a CDN, and it can take an amazing amount of load. Varnish needs memory, but without Varnish we could run this setup on a box a fraction of the size.

This plan costs $19/month! I still can't believe we're paying so little.

Instead of focussing just on the server though, and like the TumbleDry article somewhat suggests, HTTP cache is probably the best place to start in terms of performance. Varnish, CDNs, etc all rely on intelligent HTTP caching. And if you do it right, you don't need to worry (too often) about cache invalidation.

What I'm really looking forward to is making use of ESI in Symfony2 and Varnish. That will mean setting different cache headers for portions of pages, which will further reduce the need to manually invalidate cache.

For now though, I'm loving Nginx + FPM + APC.

ez772• 15 years ago

For static, high-traffic, small-size content, doesn't it make sense to load a minimal OS entirely to RAM and serve it from there? Has anybody tried this? (I guess this rules out VPSes...)

Note: This is a variation of a previous comment I made, but a variation nonetheless. Sorry to belabor the point.

drv• 15 years ago

The title says "120 megs of RAM", but I wonder if that's at all comparable to a real machine with 120 MB. I imagine that the "120 MB" VM is running on a beefy host with tens (or hundreds) of gigabytes of RAM shared between the guest VMs and also used for disk cache. It seems likely that accessing a guest's virtual disk would actually hit the host's disk cache a lot of the time (especially when that guest has been busy recently); that would improve the speed of disk access for that VM enough that it could make up for the lack of memory for disk cache within the guest.

This is purely speculation, but I would be interested to see if there is any actual research to back it up.

I suppose if the guest in this instance is not swapping very often, then this is fairly irrelevant, but the article didn't mention anything about swap.

9 million hits/day with 120 megs RAM

Related Stories