Paul Buchheit: Make your site faster and cheaper to operate in one easy step

http://paulbuchheit.blogspot.com/2009/04/make-your-site-faster-and-cheaper-to.html

by paul • 17 years ago

186 59 17 years ago

paulbuchheit.blogspot.com

Ardit20• 17 years ago

Umm I am confused. Is this about compressing the web page itself? I think what takes most of the time to load on my website is the advertisers rather than the content itself. So perhaps this is about large websites like amazon which have large detabases?

ulf• 17 years ago

I think what he means is just enabling transportation of compressed data from server to client, regardless what you are transporting. The size of your page does not affect the benefits you achieve with this method, which is always less data being transported.

lucumo3• 17 years ago

GP is right about advertisers though :-(

tlrobinson2• 17 years ago

Gzipping is especially important for large JavaScript heavy apps. 280slides loads about 5x faster gzipped than not.

If you're really worried about the performance hit of gzipping, you can cache gzipped versions of static resources.

aristus2• 17 years ago

How do you deal with the time it takes to parse a large Javascript file? Past 500KB or so it can take several seconds.

enomar2• 17 years ago

Are you minifying your javascript?

500KB is pretty huge. Does every user need that? Perhaps you can use a smaller bootstrap script to pull down only what's needed when it's needed.

prakash2• 17 years ago

One scenario where using gzip might not be a good idea is when serving content less than 2-4 kB, like some thumbnails.

chaosmachine• 17 years ago

Gzipping images is generally not a good idea. PNG/GIF/JPEG are already highly compressed and will probably grow in size if you attempt to gzip them.

_pius• 17 years ago

PNG is often not well compressed unless you've gone out of your way to do so with PNGCRUSH (http://pmt.sourceforge.net/pngcrush/) or something similar

eru• 17 years ago

When you serve thumbnails in an already compressed format like PNG, JPEG, GIF, then it should not make any sense to gzip them anyway.

prakash2• 17 years ago

true about the image part. images shouldn't be gzipped, bad examples. It still stands for files less than 2-4 kB.

mkinsella• 17 years ago

Can Apache/nginx be configured to not gzip images or files smaller than a certain size?

Confusion• 17 years ago

His speed estimates are wrong, as he should have tested gzipping many small files instead of gzipping one large file. There is a significant difference.

nostrademons2• 17 years ago

Why? If you're setting up your website right, you should be serving 1 HTML file + 1 JS file + 1 CSS file + 1 image for each request, and the latter 3 should all be cached indefinitely. (At least for static chrome - if you've got thumbnails, videos, or flash games you can't exactly sprite those.) His test file was 146K, which seems on the high side for HTML only, but I'd imagine serving 1 small file is faster than serving 1 large file.

stcredzero• 17 years ago

1 image for each request? Are you putting all the images for each page into one image file, then cropping the needed region for each image on your page?

paul3• 17 years ago

Have you actually made any measurements?

Smaller pages are actually faster (both individually and collectively), though gzip is already fast enough that the difference is irrelevant.

andr• 17 years ago

That's only one of the needed steps. Adjust your headers so that static items are not reloaded frequently. Use versioned urls (e.g., /scripts/main.js?234) and update the versions only when you change the scripts or CSS. YSlow goes a long way in helping with that kind of stuff.

The same applies for S3 uploads. You can pass cache headers in the upload request which will later be used on all downloads.

ivankirigin• 17 years ago

Do you know something that can modify django templates that contain CSS & JS imports so that the linked version gets updated with each checked in change to the files?

lethain• 17 years ago

django-compress does this very well (integrates with YUI Compressor, and several other compression filters). I use it on all my django sites.

ulvund• 17 years ago

a2enmod deflate

/etc/init.d/apache2 restart

bkudria• 17 years ago

Thanks. For some reason, this critical bit of information was missing from the AppJet app, the article, and the comments. I had to spend some of my time actually looking for it.

paul3• 17 years ago

I don't use apache, but apparently it may be a little more complicated than that if you really want it to work: http://www.nerdblog.com/2009/04/my-modgzip-settings-deflatec...

sayrer• 17 years ago

Indeed.

http://www.ilikespam.com/blog/internet-explorer-meets-the-va...

mnemonik• 17 years ago

Wow, thanks! For fifteen minutes I've been scratching my head after I read all the docs on deflate. Nowhere was that command mentioned and when I saw your comment I had an "Of course!" moment. Thanks for that.

axod4• 17 years ago

Mibbit uses a custom webserver... Instead of gzipping things on the fly, I decided to just look for a .gz version on the filesystem, and use that if it's there.

eg a request for 'index.html' looks for 'index.html' and 'index.html.gz'. If the gz is there it uses that and sets headers accordingly.

Works incredibly well, and the deploy scripts just gzip things when they're pushed out to production.

emmett• 17 years ago

I don't know if this applies to your site or not, but if you have any dynamic http requests (non-cacheable) that approach doesn't work very well. You're much better off doing it on the fly with nginx as paul suggests; the impact on CPU is not noticeable even under very heavy load.

axod4• 17 years ago

Yeah dynamic stuff is a whole different kettle of fish. The dynamic content on Mibbit is usually very small - maybe 100-200 bytes. The HTTP request headers are usually more (Which can't be compressed).

For Mibbit, I'd like to eventually do my own compression which will beat the socks off gzip, as it'll be session based rather than per request.

But yeah, I can see use cases where you're sending dynamic stuff which will benefit from gzip and where pre-caching on disk doesn't really make sense.

zepolen2• 17 years ago

Out of curiosity what was the reason for using a custom webserver?

axod4• 17 years ago

Scalability, full control over everything, it's not rocket science...

Mibbit uses some cool Comet like stuff, and I'm 99% confident the Mibbit webserver is better than anything else at doing this.

varun2• 17 years ago

For AJAX-based web applications, following is what I'd suggest in making things very zippy for the user:

1. Concatenate your JS and CSS files. Don't send out several files over the wire to the browser - the browser can only make 2 connections at a time. Be careful about JS dependencies - order is imp. in JS.

2. Minify and then compress the JS and CSS. Use Dojo's Shrinksafe or the YUI Compressor to do this. It will strip out whitespace, etc - make the code smaller in size (In JS, every byte counts) and compress.

3. Now gzip the above. (Paul's article talks only about gzipping - if you do the above 2 steps as well, you'd improve the performance a lot more).

Write an Ant script to automate all the above on code commit and you are done. Try other methods like loading other elements in the background or after a tab etc is clicked - important to show something to the user almost instantly. Did this for Alertle.com, which was a 100% AJAX web app (no page refresh at all), and the initial size of the code being sent to the browser went from 700k to about 20k using the steps above :)

uggedal• 17 years ago

While it's true that IE6 and IE7 can only handle 2 concurrent persistent connections per server, IE8 can handle 6.

Firefox 3 by default handles max 8 persistent connections per server, and max 15 connections per server in total (persistent and non-persistent).

This goes against RFC2616, but I guess the capacity of both servers and clients have increased enough the last 10 years to warrant such changes in default behavior across browsers.

kragen3• 17 years ago

The RFC2616 requirement was always a bad idea for users; for years I missed the Netscape feature that let you set this parameter to whatever you wanted; I left it at 20. (Was that up to 0.91N? I forget.) It helped out server software that made concurrent connections expensive, though.

zepolen2• 17 years ago

> the browser can only make 2 connections at a time

This is slowly becoming not true anymore, decent browsers like Opera and Firefox have defaulted to 8 for a while now, and IE8 defaults to 6.

Although all your points are still valid.

litewulf• 17 years ago

Another good argument in favor of doing so is that establishing a connection has a non-zero time cost. Further, your server may not have the workers to spare (so that extra connection is going to sit until the server's queue isn't backed up.)

varun2• 17 years ago

4. Figure out browser caching - what to cache, how and for how long. Frequently changing code files - not a good idea to cache them for long, but images and other files, in most cases you only really need to download them once to the user's browser. Stuff to Google: etags and last-modified.

Improving site speed is a broad topic, and there would be other stuff on the server-side too where improvements can be made, like cached queries and using "prepared statements" to optimize the SQL.

enomar2• 17 years ago

Check out YSlow and Steve Souders' tips if you're interested in speeding up your front end.

http://developer.yahoo.com/yslow/ http://developer.yahoo.com/performance/rules.html

dbul4• 17 years ago

Blog entries like this make me feel truly educated. I could visit my grandparents in Weston, MA and regurgitate, "the highest merit we ascribe to Moses, Plato, and Milton is, that they set at naught books and traditions" and my grandmother would reply, "Ah, someone who is educated!" Whereas if I had said, "GZip encode your hypertext," she would be the uneducated one.

So education is relative. Perhaps if I give in and move to the Bay Area my education will be richer than I would ever have imagined.

allenbrunson• 17 years ago

i don't think i understand your comment, but it was pretty entertaining, nonetheless.

dbul4• 17 years ago

It was casual. But truthful in sentiment. You can see it as ironical in that I was educated by the Internet and not by being in the Bay Area. Yet, being in the Bay Area, making the right friends, and appropriately asking the right questions and exchanging ideas may prove more useful than the Internet, sometimes. (After all, the Internet is still by your side if you need it.)

bestes2• 17 years ago

Nobody has mentioned the latency issue when gzipping. If you have to construct the whole file before gzipping, in situations where the file is large or dribbles out as the server processes the data, this could mean a significant slowdown. In virtually all situations, I agree gzipping is good, just like I always leave write-caching on my hard drive turned on so that the slowest part of my system can run at the fastest possible speed. There just might be consequences you do not intend. To address the obvious replies: yes, your server should not dribble out content. And, yes, if you are using a framework that spits out the entire page at once already, you will incur no additional latency on top of the gzip/gunzip time.

tlrobinson2• 17 years ago

For large static files you can cache pre-gzipped versions and use content negotiation to serve it to clients that support gzip.

paul3• 17 years ago

Incorrect. Gzip streams just fine, so there is no latency issue. Google search, for example, writes the top of the page before the search is complete. (and I assure you, they use gzip)

nostrademons2• 17 years ago

Thanks for saying that - I was gonna pipe up but wasn't sure if it was public information. :-)

bestes2• 17 years ago

I'm sure I had this problem before, but maybe it was with an older server or browser? In any case, thanks for the correction.

Paul Buchheit: Make your site faster and cheaper to operate in one easy step

Related Stories