Faster Firefox landing page increases download conversion by 15%

http://blog.mozilla.com/metrics/2010/04/05/firefox-page-load-speed-%e2%80%93-part-ii/

by mbrubeck • 16 years ago

83 53 16 years ago

blog.mozilla.com

tokenadult6• 16 years ago

It's surprising it would take people with their background so long to figure that out.

wglb2• 16 years ago

Yes, but as seen by lots of recent HN postings have shown, actually using a/b tests seems to be a rarity.

tokenadult6• 16 years ago

My comment was more along the lines that a page that promotes your service should load reliably rapidly, because people who are kept waiting by page loading may not stay around to convert to a user of your product, even if it is free. The comparisons the Firefox team show with the webpages of Brand X products made me think "Ouch!" because I am a user of Firefox myself.

aresant• 16 years ago

You are dead on there - it's hard to believe.

And forget a/b testing - Google's Optimizer provides full multivariate testing, FREE.

We paid $10k a month to license a hosted multivariate test engine just 3 years from Optimost that was a piece of junk in comparison to that free tool . . .

ZeroGravitas• 16 years ago

The Mozilla people seem to be quite good with the stats. I think they've looked at the whole user "funnel" (i.e. from website to download to install to continued use) and focused on the parts they can get the biggest wins. Note the blog title: Blog of Metrics

Though perhaps they've got a blindspot in that they're more comfortable making changes in desktop code and so started there.

DEinspanjer• 16 years ago

Mozilla had a lot of people focused on other areas who were trying to do ad-hoc analysis on the side. That is why our team was formed. The metrics team set about to measure and improve user interaction with certain critical pages like this IE landing page, and we discovered that it hadn't received the same level of attention that certain other pages had in regards to optimization.

CytokineStorm• 16 years ago

Finally! It looks like someone actually did some simple math to determine if their results were statistically meaningful. I wish more people would include a section like this when they talk about conversion rates.

sesqu• 16 years ago

15% looks great, but how confident are we?

Running a one-sided Student’s t-test with a means difference of 14%, our experimental data yields a P value of 0.000051. This means that there is only a 0.0051% chance that we would obtain a 14% (or greater) improvement if the real effect wasn’t at least this large.

Err... Student's t doesn't assign a probability to P(measured>=14 | actual<14), it assigns one to P(measured>=14 | actual=14) under the assumption of equal variances (which here means actual=0, so clearly not applicable).

I figure that the actual effect is 16.05% +- 0.17%. And that's simply by assuming 145k is close enough to infinity, and for IE.html only.

That is, unless I'm mistaken about this, since I grok only the fairly basic stuff. Which is certainly plausible in and of itself.

robryan2• 16 years ago

Things like this always make me worry about the patience levels of some people.

tokenadult6• 16 years ago

People ALWAYS have something more important to do than wait for your site to load, always. It's not that your site is unimportant, it's that your users' personal lives are more important to each of those users. But don't take my word for this; this is a well replicated finding of usability research in hypertext environments that go back before the development of the World Wide Web.

robryan2• 16 years ago

I think they have hit the long tale of firefox browser adoption, people that don't care much what browser they use.

patio114• 16 years ago

This worked about as well for me as it did for Firefox, when I implemented it a few years ago, after reading the YSlow presentations. I've said it three times on HN but I'll say it a fourth: watch one of the YSlow presentations, get their checklist, go down and tick off items on it. It is the easiest money you'll make in your life. Look at the improvements here: combining uncombined JS files and inlining CSS. These are tweaks that you can have coded, tested, and live in ten minutes or less.

As an aside: It is a little disturbing to me that I'm ahead of Mozilla on the Internet technology adoption curve.

exit• 16 years ago

does it make sense to inline css if it's the same across multiple pages?

anthonyb2• 16 years ago

Should do, since it's one less connection/round trip back to the server to get more data.

patio114• 16 years ago

This would depend on your goals. If first load speed is paramount, for example the landing page for Firefox or on google.com, then go with the inline. If you anticipate (or, ahem, have measured) multi-page interactions from your users and can deal with a mild hit on the first page, minify+combine+gzip one CSS file.

I'm very much in group #2... everywhere except my landing pages.

chaosmachine• 16 years ago

Inspired by this comment, I just spent 8 hours re-optimizing one of my sites. I managed to cut loading time in half. We'll see if that has any effect on sales...

pg4• 16 years ago

Incidentally, how is HN for speed?

patio114• 16 years ago

"Very freaking fast."

The front page loads with 10 HTTP requests on an empty cache, and doesn't have any user-perceptible blockage while waiting on elements. (Pulled this straight out of YSlow.)

Improvement is overkill, but if you wanted to:

1) Creates images{1,2}.ycombinator.com and split y18.gif, s.gif, and grayarrow.gif, and your JS and CSS files across the three domains. This will cause older browsers to load them all in parallel. (The HTTP spec suggests having no more than 2 requests to any one domain open at a time, so you end up with stairstep loading graphs on browsers which are spec-compliant, such as IE6. Browsers which have a more pragmatic view towards compliance with published specifications, such as Firefox 3, will do 8 ~ 10 requests in parallel.)

Note the balancing act in that every domain you add is another DNS query that needs to get resolved once. Ideally you want to keep it to about 4 domains or less.

2) Put far future expires headers on your static assets. You can bust caches by using the Rails-y filename.js?timestampOfLastModificationOfCode method.

But, again, this site is probably the fastest one I use on a regular basis, even from braindead clients like my Kindle. I wouldn't guess the marginal improvement is worth the expensive engineer time.

jeff184• 16 years ago

This is good advice, but not optimal. Using multiple domains in this case will actually slow down the site because of the additional DNS lookups -- only do that if you are serving up a ton of images.

a) The CSS file should be inline.

b) The images should be done away with entirely by including them as a data uri.

c) For IE6-7 (data uris not supported) the fallback image should be sprited.

d) For bonus points (if you want HN to be served nearly instantly, globally) splurge for an application caching CDN provider like Akamai.

cmelbye2• 16 years ago

The backend software (custom written by pg) can be buggy and slow at times, but the actual HTML interface is fine for performance.

pg4• 16 years ago

It's not that buggy, is it? Requests occasionally time out, but I don't know of much that actually breaks.

aw3c2• 16 years ago

Accessing my threads page takes many many seconds.

Devilboy• 16 years ago

Mine often times out. When I hit refresh after a timeout it loads though, I guess I'm hitting the cache the second time.

pmjordan• 16 years ago

I'm not sure the "landing page" optimisation is all that relevant to HN. However:

Aside from the threads issue everyone else has mentioned, I've noticed that apart from the main HTML, the co2stats tracker is the only part of the site that doesn't have proper cache-control headers and thus incurs a query every time (resulting in a 200 OK, not a 304 Not Modified, say); worse, the javascript portion can also take a while to load (up to 2 seconds), even though the returned content never actually seems to change. The co2stats image also never is cached. You should probably prod them to fix that (they're YC funded, right?).

axod• 16 years ago

When logged out, it's fast.

When logged in, most things take a few seconds to load. I find myself opening things in new tabs and waiting to come back to them.

Threads, comments, all take a long time to load and often timeout.

Making a mistake in a comment costs 5-10 seconds, since you'll have to click on [edit].

Usable, but only just.

wglb2• 16 years ago

For most of the uses for me it is a good speed. However, there are a few activities that during most times of the day time out. If I look to see a user's submissions, that will frequently time out. Occasionally, checking on my 'threads' will also time out.

houseabsolute3• 16 years ago

I don't know if this talk was ever made public but I heard at Google that under some circumstances cutting a few hundred milliseconds of latency can double click-through, conversions, etc., or even more. I do occasionally see Google people encouraging web developers to make their pages faster at conferences and such. Personally I would be very concerned about the money I was losing if I had a web site where the 95th worst percentile of my paying customers had to wait more than two seconds (from the client's perspective) for a pageload. I'd buy a bigger database machine, spread my servers to more geographical regions, whatever it took to try to bring that latency down.

patio114• 16 years ago

Google's Marissa Mayer has presented their page loading results a few times publicly: in one unplanned experiment, about .5 seconds in marginal load time caused number of user searches (and, by extension, clicks on ads and Google revenue) to decline by 20%. That was the Web 2.0 Conference in 2006.

I'd buy a bigger database machine, spread my servers to more geographical regions, whatever it took to try to bring that latency down.

Why make life hard. Buying a bigger database server costs actual money and time. Getting out the YSlow checklist and knocking two to three things off of it costs nothing and no significant amount of engineer time.

Gzip CSS/JS files: four lines in your Apache config, warm restart, done. Collapsing CSS/JS together: 16 characters (:cache => true) in Rails, warm restart, done. Spriting CSS images: open a web page, type a bit, copy/paste what it tells you to to development, push to staging, verify it works, done.

Moreover, as compared to buying a DB or spreading your servers, these are virtually guaranteed to actually work. Many, many of the suggestions that come under the heading "performance improvements" do not actually work because they address things that are not problems. (For example, unless you have data which convincingly demonstrates differently, the vast majority of websites can assume that the web stack is not a problem. Optimizing your code within the web stack is generally a hideously expensive waste of time.)

houseabsolute3• 16 years ago

Sure, fixing the webpage would come first. I only mean to demonstrate that I would spare no metaphorical expense in fixing the problem.

iigs• 16 years ago

It's clear that a benefit was seen. Moreover, it's clear that in other cases (shopping at Amazon, searching at Google), more responsive servers mean an increase in traffic, since people are free to shop / browse more, and it keeps people's interest.

In the case of a single site with a single product / purpose (ignoring Mozilla's other products), is there a generally agreed on explanation for what is going on here? Do people really change their mind about something as big as changing their web browser because of a delay of one-two seconds? Are these people marginal users and unlikely to actually finish installing it, or likely to abandon the browser after a single use?

I'd like a peek into the psychology of the marginal people in a scenario like this.

tokenadult6• 16 years ago

I often conclude that the people running websites are doofuses, and thus their product may be a no-good product, if the website doesn't meet the universal usability standard of speedy page views. That's a simple quality proxy heuristic.

Faster Firefox landing page increases download conversion by 15%

Related Stories