The average size of Web pages is now the average size of a Doom install

https://mobiforge.com/research-analysis/the-web-is-doom

by Ovid • 10 years ago

857 447 10 years ago

overcast2• 10 years ago

With the majority of users moving towards mobile, I really think this is an issue, and I've been consciously building projects as lean as possible. Removing bloated jquery libraries was a big one. With native calls like document.querySelectorAll document.querySelector I've found I can 90% get by without it. For the rest, using something like vue.js, and I've taken care of all the dom manipulation, data binding, etc.

Ruud-v-A2• 10 years ago

How about not using js at all? I think that for the majority of pages it doesn’t add any value.

overcast2• 10 years ago

With the instant gratification generation, not having dynamic content is going to be a tough sell. I will however agree, there is a gross overuse of it.

smrtinsert• 10 years ago

I really wish more companies cared about their mobile presence. You don't need much, especially since native mobile widgets go so far along towards making web a nice thing. Also, for gods sake allow zooming. I think of that as such a major plus for mobile and yet so many mobile sites disallow it.

CaptSpify5• 10 years ago

I lived in $nowheresville, TN for a bit. This drove me insane. Sure, your webapp works great in downtown SF, but try loading it with the spotty connection the rest of the country has, not to mention the rest of the world.

One of the places I worked at had a computer hooked up to a ~800 kbps modem, and would test all their web-pages on that. It was really eye-opening, and I wish more companies would benchmark like that

mod2• 10 years ago

Chrome network tools has throttling that can simulate really shitty networks, I use that a fair bit.

collyw• 10 years ago

I remember thinking years ago, that my CV in Word took up more memory than my first computer (Acorn Electron 32kb ram). It amazes me that I used to play Elete on that machine.

warriorkitty2• 10 years ago

Oh, you just want to add a class to the element? \adds whole jQuery\ That's what's wrong with the web.

Oh, and you need a loop? \adds underscore.js\

rplnt2• 10 years ago

The community is the worst. The adds whole jQuery comes from every single js topic on SO in the past five+ years being answered by "just use jquery". Many times they are not even web related at all.

"How do you do something in javascript?"
"With jQuery you do it like this..."

I had the worst time ever when I had to work with jscript. I really wonder if my dislike of the language comes from the language itself or the community around it.

boredpudding• 10 years ago

A long time the web was like this:

In IE6, you do it like this.

In IE8, you do it like this.

In Firefox, you do it like this.

Then jQuery came along. And it was like 'Now you do it like this, and jQuery handles it for all browsers perfectly'.

Just because some tasks are now performed easily in native javascript on all browsers, doesn't mean it was always that way.

megalodon• 10 years ago

SO feels more like a race to answer fast rather than to answer with quality. I think the best way to learn about the language is through the Mozilla Developer Network. I can't speak for everyone but I feel incredibly productive when programming in javascript.

I'd argue that jQuery was great, but now most (if not all) of it can be replaced by native javascript features (e.g. document.querySelector). Today I wouldn't recommend jQuery to anyone.

I suggest learning a functional language (in my case it was Haskell) in parallel, as it opens up new ways of thinking about javascript and problem solving.

CaptSpify5• 10 years ago

correct! reminds me of: http://i.stack.imgur.com/ssRUr.gif

I think SO has gotten a little better about this, but not much, IME

nailer• 10 years ago

I totally agree - I love raw JS too, and I hate seeing underscore used instead of stdlib - but the stdlib and DOM API has been getting better very slowly:

- element.classList is relatively recent

- NodeLists still don't have .forEach()

- string.includes() and array.includes() was only added in ES6 and ES7

- As of a Chrome 50 a few weeks ago you can now see the values inside FormData.

Retric2• 10 years ago

jQuery is not that large and very likely to be cached an a users machine. So, it's IMO disingenuous to harp on it.

If your playing with larger and less 'standard' library's then that's a different story.

Conlectus2• 10 years ago

There are approximately 5 million different CDNs hosting different versions of jQuery. Cache hits are actually quite low.

shawabawa33• 10 years ago

It would help if the basic javascript api's were even vaguely useful

I recently tried to force myself not to use any libraries for a simple site. After a while I realised I had so much re-invention of stuff (AJAX in particular is madness without a library) that I ended up adding Zepto

With all the crap they're adding in ES6 you would have hoped they would add an ajax function at least

Conlectus2• 10 years ago

it's called the "fetch" API. Not sure if it's in ES6

joshuacc• 10 years ago

Neither XMLHttpRequest, nor the Fetch API mentioned in a sibling comment are part of the JavaScript language. They are both web platform specs maintained by the WHATWG. The JavaScript committee (TC39) does not control these platform APIs.

spiderfarmer4• 10 years ago

A lack of skills, a low barrier to entry and low budgets are causing this.

I hoped with page performance becoming a ranking factor, this would change, if only for a tiny bit. But I see very slow brand websites still ranked higher then the highly optimized indie website, so that didn't work.

swalsh• 10 years ago

Imagine how fast the web would be if browser manufacturers could "approve" frameworks, and store them as a "standard library" of sorts. A global cache. Then you have ONE copy of jQuery for all sites on your machine.

Wanting to use a framework shouldn't be discouraged. The issue here isn't developers trying to save time, the issue is the shitty tools.

adrianN2• 10 years ago

Look how long it took for browsers to kinda agree on how to interpret Javascript and HTML. Do you really want to add another way for browsers to behave differently?

toyg2• 10 years ago

Yeah, javascript historically lacks a stdlib and now it's so big that nobody could probably agree on one. Chalk another one for "the road to hell is paved with good intentions".

NoGravitas2• 10 years ago

This is basically what [Decentraleyes](https://github.com/Synzvato/decentraleyes) does. It helps especially on mobile, though it's not exactly life-changing IME.

tedmiston3• 10 years ago

I wish this could be possible.

Eventually you'll have to track every minor version of every library when not every site stays up to date. At that point it's (almost) effectively a CDN.

Houshalter• 10 years ago

But shouldn't the browser have those libraries cached?

dagw• 10 years ago

Given the number of combination of CDNs and jQuery versions out there I'm guessing the chances of already having the 'right' jQuery in cache the first time you visit a site is quite low. Would be interesting to see some numbers though.

VeejayRampay2• 10 years ago

Well, not that adding the whole of underscore.js is reasonable, but maybe if Javascript had provided proper enumeration of their data structures earlier like all decent programming languages do, that wouldn't be an issue.

To this day, you still can't enumerate an object literal, yielding each key/value pair. You still can't properly enumerate a NodeList and Object.values is still "experimental technology".

I understand that people just abuse libraries in and out, but come on, let's not forget the reason those humongous libraries exist in the first place (and why they're having success success), it's because JS has long been crippled in terms of abstractions and people needed something to alleviate the pain.

EDIT #1: Regarding enumeration of object literals, it turns out it's actually possible, though Object.entries is still marked as "experimental" and returns an array of [key, value] pairs, as arrays, which doesn't exactly scream great design. I just wish I could do something like someObject.forEach(function(key, value) {}) (and same for map, reduce, filter, some, every, etc).

EDIT #2: The latter in my previous edit is actually possible if you're willing to create a Map from Object.entries, which you can call forEach on. Quite convoluted, but possible, so my bad.

hzoo• 10 years ago

Yeah, there will be Object.keys/Object.values/Object.entries and you can use it in an for of loop.

for (let [key, value] of Object.entries({a: 1})) { console.log(key, value); }

seangrogg2• 10 years ago

var obj = {...}

// native JavaScript

Object.keys(obj).forEach(function(key){ console.log('obj.', key, ' = ', obj[key]) })

// Lodash

_.forEach(obj, function(value, key) { console.log('obj.', key, ' = ', value) })

Object.keys(obj) creates an array that contains the keys of the object, which makes it pretty trivial to call filter, map, reduce, etc. And getting the value from an object's key is also pretty straightforward...

onion2k• 10 years ago

Element.classList doesn't work in IE9 or below.

Array.forEach doesn't work in IE8 or below.

Fetch API for ajax isn't in any version of IE or Safari, or any mobile browser apart from Chrome for Android.

And so on.

It's not so much that front end devs are lazy, but more a case that building a set of polyfills for each and every browser support problem is actually hard. "just use jQuery" gets around the problem. I think most developers want to write better code, but most projects don't have the development bandwidth for them to do things better.

hawski• 10 years ago

I am not JS dev, but... At this point there should be tree shaking / dead code removal for JS widely deployed. Why it is not? I know that dynamic nature of JS causes some of it, but most code out there is not that dynamic. How good is Google's Closure compiler?

lloyd-christmas• 10 years ago

rollupjs.org as well as webpack2 are doing this. As you said, the dynamic nature will prevent this from being native. Using a build tool isn't terrible though.

cm32• 10 years ago

If instead of JavaScript a more fitting domain language was used for scripting web apps, this would be less of an issue.

lhorie• 10 years ago

These days, for simple interactions, youmightnotneedjquery.com is a good resource for those trying to curb that sort of behavior.

On the framework end of the spectrum, we have efforts like Mithril.js that try to provide a minimalist set of tools for more ambitious web applications.

So all is not lost :)

sp332• 10 years ago

I don't think jQuery is the problem. jQuery is not 1.2 MB. It's not even 0.1 MB.

jmnicolas• 10 years ago

The average size of Web pages is now the average size of a Doom install

It's not really surprising in a world where a graphical driver is > 100 MB (Nvidia driver for Windows).

warriorkitty2• 10 years ago

Oh come on, let's talk about Adobe Reader and Skype. :)

elsurudo• 10 years ago

They mean the original Doom, which ran with a software renderer anyways.

batat2• 10 years ago

And 43 MB chromium mini_installer.exe

throwanem• 10 years ago

Only a hundred megabytes? You're getting off light there. I went to install the latest Nvidia CUDA driver the other day, and it's a gigabyte download.

ebbv2• 10 years ago

I would argue two things:

1) This is an irrelevant statistic.

2) Even if this were true it's not that big of a deal.

This is irrelevant because most people don't browse the average web page. They browse the top few sites on the internet and that's it. A more relevant statistic would be what have the sizes of the top 50 sites been over the last 15 years. I imagine they still may have grown on average, but download speeds have also grown over that time. Especially on mobile.

Even if we accept the premise that web sites as a whole, including the most popular ones are all growing and are now an average of 2.2MB each. Who cares? 2.2MB is nothing in 2016. Even on an LTE connection that's probably between 4 and 1.5 seconds to download the full page. And a lot of that size is probably in ads, which nobody minds if they load last or not at all.

Lastly, this is a self fixing problem. If a site is too bloated, users will stop going to it.

But I would propose that a lot of this increase in size is due to users (especially mobile) having higher and higher resolution displays, which necessitates higher resolution content, which of course is bigger.

Ruud-v-A2• 10 years ago

I care. I know a lot of people (here in western Europe) who are on a 1 GiB/month data plan. 2.2 MiB per page means that you can visit 15 pages per day. Better think twice before you click a link.

Besides, 2.2 MiB for a page is pure bloat. Unless the page is heavy on images, you can usually fit all of the content that matters in a tenth of the size. In 300 KiB you can fit a 500-word article with 160 KiB picture, a few webfonts, headers/footers and stylesheets. Using a factor ten more is just ridiculous.

niutech11• 10 years ago

I would recommend using Opera Mini or GDrives Web Light (http://gdriv.es/l/)

radiospiel• 10 years ago

The reality, however, is that depending on where they are many users don't have an LTE connection to begin with. My personal experience is that every so often a page begins to load, then shows text invisibly, then loads ads, and only after around 10 secs, I guess, I see the content I was interested in in the first place. Well, usually I closed my browser by then already.

manigandham• 10 years ago

Who cares? Everyone does. Everyone wants faster sites and downloading 2.2MB is not easy or quick.

It's a ridiculous amount of data needed to display sites that don't have any special media or functionality and it burns up battery life and data plans for most mobile users.

Also latency is a far bigger effect than bandwidth when interacting with sites so I would say even with bandwidth going up, the actual experience of loading a site has gone down considerably over the last few years.

jokoon4• 10 years ago

Internet infrastructure is far from being cheap, and CPUs are not free either.

Who cares? 2.2MB is nothing in 2016.

Multiply that by the amount of internet users, and now try to imagine all the hardware running this.

It's also about having a format that discourage bloat so that the web can be faster on a large scale.

ebbv2• 10 years ago

What? The size of a page has nothing to do with the CPU. Most of that size is going to be in images and animation, and most web pages put basically no load on your CPU. If one does it's going to be because of badly written JS, not because of the size.

Shrinking the average page size even by half is not going to "the web run faster on a large scale." Most of the time your page load speed is not affected by congestion, and if it is it's likely due to a local tower or something. The problem there will be the number of users per tower, not the size of the pages they are loading.

detaro6• 10 years ago

Even on an LTE connection that's probably between 4 and 1.5 seconds to download the full page

So on the fastest mobile connection type available it's a magnitude or more to slow to feel fast, and that's a good state of things?

pljns2• 10 years ago

The average Web page now does more than the average Doom install, I don't see the relevance of this.

Although I get really annoyed when I visit a blog post whose page is 100x larger than Dostoevsky's novels in .txt format. On my blog (https://pljns.com/blog/), JQuery and genericons are often my largest file transfers, but I still clock under 500kb.

blub• 10 years ago

Displaying text on screen with some crappy ads & animations is not doing more than Doom, not even close.

Marazan• 10 years ago

Once you remove Ad Tech how much does the average we page actually do?

batat2• 10 years ago

https://pljns.com/

40-pound jQuery file and 83 polyfills give IE7 a boner because it finally has box-shadow

[x] check

You loaded all 7 fontfaces of a shitty webfont just so you could say "Hi." at 100px height at the beginning of your site?

[x] assuming 404'ed fontawesome as shitty webfont, check

You thought you needed media queries to be responsive, but no

[x] check

Your site has three bylines and link to your dribbble account, but you spread it over 7 full screens and make me click some bobbing button to show me how cool the jQuery ScrollTo plugin is

[x] check

Still pretty good site, but it's funny how accurately creator of http://motherfuckingwebsite.com/ has described the situation with the modern web :)

pljns2• 10 years ago

These are all good points that I knew when I hastily pushed the site last week. Still way under 1000kb!

Also, I said my blog, but serves me right I guess ;-)

jessriedel3• 10 years ago

I'm skeptical that developers talking to each other about how bad web bloat is will change anything. They will still face the same incentives in terms of ad revenue, costs of optimization, etc.

Here's a random idea that might have more potential: create an adblocker browser plugin that also colors URLs based on how slow they are expected to load, e.g., smoothly from blue to red. The scores could be centrally calculated for the top N URLs on the web (or perhaps, an estimate based on the top M domain names and other signals) and downloaded to the client (so no privacy issues). People will very quickly learn to associate red URLs with the feeling "ugh, this page is taking forever". So long as the metric was reasonably robust to gaming, websites would face a greater pressure to cut the bloat. And yet, it's still ultimately feedback determined by a user's revealed preferences, based on what they think is worth waiting how long for, rather than a developer's guess about what's reasonable.

mtgx• 10 years ago

I also like how Opera now allows you to benchmark websites with and without ads. I think other browsers should expand on that idea (benchmarking sites, and not just for ads).

Give some kind of reminder to both users and developers about how slow their sites are. Those with the slowest websites probably won't like it too much initially, but it's going to be better for all of us in the long term.

http://www.opera.com/blogs/desktop/2016/03/native-ad-blockin...

d333• 10 years ago

I would say that a possible solution is also to better rank websites that mention the checksums of their external resources and make web browsers keep them in cache much longer... if pretty much every website uses jQuery, perhaps we should ship jQuery with the web browser?

Xophmeister• 10 years ago

Common libraries and frameworks are usually loaded from a CDN. These should stay in the browser's cache for some time.

kevincox5• 10 years ago

I hear this argument a lot, and I very much disagree. Now you have browser vendors having to device which libraries are "popular" and shipping them in the initial download of the browser.

It turns out that this technology already exists in a much better form. It's called cache. The problem is that almost everyone hosts their own version of jQuery. If everyone simply linked the "canonical" version of jQuery (the CDN link is right on their site) then requiring jQuery will be effectively free because it will be in everyone's cache.

Also the cache is supported by all browsers with an elegant fallback. Instead of having to manually having to check if your user's browser has the resource you want preloaded you just like the URL and the best option will automatically be used.

TL;DR Rather then turning this into a political issue stop bundling resources, modern protocols and intelligent parallel loading allow using the cache to stove this problem.

illumen• 10 years ago

There are checksums available for resources now. As of last few months. Also things like jQuery are available from multiple CDN, and they are used by many of the top websites and html frameworks. So you probably already have it in cache.

JustSomeNobody4• 10 years ago

We absolutely, most certainly, should NOT ship libraries with browsers. This is a horrible idea. Browser vendors would then have too much influence on development libraries.

rmc• 10 years ago

if pretty much every website uses jQuery, perhaps we should ship jQuery with the web browser?

There's a firefox plugin that does just that! DecentralEyes - https://addons.mozilla.org/en-US/firefox/addon/decentraleyes...

Although mostly for privacy reasons, so google doesn't know all the sites you visit.

douche• 10 years ago

Hmmmm, or javascript could be standardized on something that isn't a barely-working kludge with no batteries included? If something like jQuery or Lodash is basically table-stakes for doing anything really worthwhile, then just maybe that de facto standard should be made de jure.

achairapart• 10 years ago

Why only jQuery, anyway?

What if browsers shipped with a repository instead of a cache? Download once, stay forever.

lostgame2• 10 years ago

Isn't this what the '.min' versions of these is for?

jquery-2.2.3.min.js is only 86KB for me. For the amount of functionality it adds, sure seems like a sweet deal.

Part of the problem (IMHO) is the growing requirement for larger, alpha-channel-using image formats like PNG, that then need to be expressed in a Retina format - I mean, shit looks terrible on my retina MacBook Pro that isn't properly Retina formatted. (Here's looking at you, Unity3D...even the current beta which supports Retina displays is extremely unstable and half the graphics (even stupid buttons on the UI) are still SD...)

With bandwidth costs declining [1] and basic RAM increasing [2] is there a particular reason a Web application should be much smaller than a typical desktop application? We have caches for a reason.

[1] http://www.networkworld.com/article/2187538/tech-primers/exp...

[2] http://en.yibada.com/articles/118394/20160422/macbook-air-is...

ma2rten• 10 years ago

I thought Google already punishes slow loading websites.

jessriedel3• 10 years ago

When Google does this they are explicitly making a trade off between relevance and speed on behalf of the user. I'd rather leave that choice to the user, especially when the user can infer that this is the correct destination from the search result text.

Also, it doesn't work other places.

StavrosK3• 10 years ago

People have an innate metric for "this thing is slow", it's when lots of time passes between its start and end. Why is color-coding more evident than our own perception of time?

jasode• 10 years ago

I think his key phrase is "expected to load," which means the color coding would provide an indicator before the user clicks on the links. Therefore, the user skips the unnecessary pain of perceiving the slowness for himself.

fudged71• 10 years ago

The problem is the same as "Downforeveryoneorjustme". Just because a site is loading slow once or twice doesn't necessarily mean that it's the site's fault, it could be any number of issues. But by crowdsourcing this data the truth is revealed.

wnevets• 10 years ago

Google is already starting to base rankings on page speed, thats enough to get companies to spend money on page speed.

mirimir3• 10 years ago

Yes, this would for sure be valuable.

Also, browsers could default to http://downforeveryoneorjustme.com/ for sites that loaded too slowly. AdBlock Plus and NoScript speed loads greatly. Maybe browsers could do triage on sites that load too slowly. Perhaps switch to reader mode or whatever.

tyingq3• 10 years ago

I think a plugin that enables the chrome "regular 2G" throttling (developer tools / network tab), permanently, for anyone that works in marketing or web development might help :)

Edit: Joking aside, that throttling feature is a nice easy way to let a dev team, or business counterpart, see what their site is like for say, a customer with a low-end DSL connection: https://developers.google.com/web/tools/chrome-devtools/prof...

Joeri• 10 years ago

I have to think that the devs want to make the page lighter, but their managers keep pushing more tracking and ad networks at them that they're forced to integrate.

logotype• 10 years ago

No need for a plugin if you are running Linux or OSX:

# Bandwidth trottling, enabling 150kB/s on port 80

sudo ipfw pipe 1 config bw 15KByte/s

sudo ipfw add 1 pipe 1 src-port 80

# Bandwidth trottling, disable

sudo ipfw delete 1

More useful stuff here as well, please star the repo :) https://github.com/logotype/useful-unix-stuff/blob/master/us...

Pxtl• 10 years ago

To me the bigger problem is the javascript bloat making it crawl on a mobile device with 1Gb of ram and a 1.3 ghz processor.

bobwaycott2• 10 years ago

Sounds like the perfect addition to corporate provisioning policies/profiles for all their machines!

vmarsy• 10 years ago

the Disconnect Me (https://disconnect.me) add-on is doing something sort of similar. It will try blocking the useless external content (tracking, etc.) and will show you how much your loading time decreased and how many KB you saved by not loading those.

yxlx• 10 years ago

Penalize any historically slow loading (for some arbitrary cut-off value, say 600 ms) site client side with an additional 10 seconds of waiting before any request is sent. This way users will hit the back button well before any data is sent to the server and the site operators will see traffic plummet.

If any browser vendor implemented my proposal, their users would switch to another browser. If it was an addon, only a handful of people would use it so there would be no impact. If it was done on the network level of a corporate business, users would complain. Still, one can dream :)

alexpersian• 10 years ago

Something that would be more effective is if Google began factoring page load speeds in their rankings just like they did for mobile optimization.

IMTDb• 10 years ago

They do

PretzelFisch2• 10 years ago

Why would that matter? If the website has the content I need it doesn't matter that this other site that doesn't match my query as well but loads faster is now the first result.

hippich• 10 years ago

i think they already do it.

Razengan5• 10 years ago

I think a more radical departure is needed. It's about time to acknowledge that the web is increasingly being used to access full-blown applications more often than "webpages."

Web browsers have more or less become mini operating systems, running elaborate virtual machines. There's way too much complexity for everyone involved — from web devs and browser devs to the users and the people who maintain the standards, then there's the devs who have to make native clients for the web apps — just to deliver products that don't have half the power of OS-native software. Everyone has to keep reinventing the wheel, like with WebAssembly, to fix problems that don't have to be there in the first place, not anymore:

Thanks to smartphones, people are already familiar with the modern concept of the standalone app; why not just make downloading an OS-native binary as easy as typing in a web address, on every OS?

Say if I press Cmd+Space on a Mac and type "Facebook" in Spotlight, it immediately begins downloading the native OS X Facebook app. The UI would be described in a format that can be incrementally downloaded, so the experience remains identical to browsing the web, except with full access to the OS's features, like an icon in the Dock and notifications and everything.

TL;DR: Instead of investing resources in browser development, Android/iOS/OSX/Windows should just work on a better standard mechanism to deliver native apps instead.

SonicSoul• 10 years ago

What constitutes "way too much complexity" ? What if browsers are evolving with whatever resources are available to them at just the right pace. Why would you want the browser to be hindered by an arbitrary speed/resource limit. Let it soar to the sound of fans going full speed!

khedoros2• 10 years ago

That sounds ambitious, interesting, and like a lot more work than keeping a webpage running. My first thought is "cross-platform nightmare". My second thought is that they'd have to rethink their expectations of continuous deployment/release and a/b testing.

We're either looking at making individual pages maintain binaries for the platforms they support (implying support of only those platforms that make sense to the site) or some kind of compilation framework running on the local machine.

abiox• 10 years ago

Android/iOS/OSX/Windows should just work on a better standard mechanism to deliver native apps instead

one might imagine that after these competing and incompatible native apps become a headache for crossplatform pursuits, a new platform will emerge that provides a uniform toolset for developing (mostly) native-platform independent applications.

perhaps this toolset will utilize a declarative system for specifying the user interface, and a scripting system that is JIT'd on each platform.

xj9• 10 years ago

In my opinion, if there is something that the web can't do as well as native it is a bug.

Web technologies can already do most of what you are proposing, including notifications. There are some performance issues, but they are well on their way to being fixed.

Aleman360• 10 years ago

Say if I press Cmd+Space on a Mac and type "Facebook" in Spotlight, it immediately begins downloading the native OS X Facebook app

That's exactly what Windows 10 does.

http://imgur.com/EqvATwL

makomk• 10 years ago

Have you looked at the size of native apps these days? They make the web look almost unbloated by comparison. (It feels like half of them are shipping an entire HTML rendering engine too.)

mirimir3• 10 years ago

Maybe because I don't want a bloody app for every site that I browse? Apps of that sort are just dumb. And huge security risks.

dasil0037• 10 years ago

I disagree.

This is a backward-looking argument which ignores the unique benefits of the web which made it inevitable that it would evolve into an application platform, regardless of how tortured the results may feel.

The web is the first truly cross-platform development environment. It is not controlled by a single vendor, and anyone implementing a new computing device must support the web (just stop for a second and consider from a historical perspective what a monumental accomplishment that is). Furthermore, it allows casual access of content and applications without any installation requirement. It comes with a reasonable security model baked-in, which, while imperfect, gets far more attention than most OS vendor sandboxing schemes. Last but not least, the web's primitive is a simple page, which is far more useful than an app as a primitive—for every app someone installs they probably visit 100 web pages for information that they would never consider installing an app for.

I agree that the web is sort of abused as an application platform, the problem is there is no central planning method which will achieve its benefits in more app-oriented fashion. No company has the power to create a standard for binary app deliverables that will have anywhere near the reach of the web. And even if one could consolidate the power and mastermind such a thing, I feel like it would run squarely into Gall's Law and have twice as many warts as the web.

PavlovsCat4• 10 years ago

It's about time to acknowledge that the web is increasingly being used to access full-blown applications more often than "webpages."

I think that is orthogonal to bloat. Sure, a complex app will always have more to load and compute than a static page with one blog post on it, but that doesn't mean an app can't be bloated on top of that, just like pages with just a single blog post on them can be bloated.

tammer• 10 years ago

Not sure about the conclusion but you're absolutely right about the problem. This is why the social networks are eating such a great percentage of screen time: pure content, outside of web applications, is better suited in a single format rather than the superfluous loud magazines that currently pepper the web.

Perhaps rather than native apps what we need is the return of gopher. I think that's what Apple's trying to do with Apple News.

tnorgaard• 10 years ago

Interestingly, this was what Alan Kay was advocating for on the web: "A bytecode VM og something like X for graphics" [1].

When Lars Bak and Kasper Lund launched Dart [2], I found it sad that they weren't more bold - left CSS and the DOM alone, and created an alternative Content-Type. So you can choose to Accept 'application/magic-bytecode' [3] before text/html, if your client supports so. Sadly, we ended up with Web Assembly, which by the few talks I've seen, appears to only cater to that of graphic/game developers, with no support for dynamic or OO languages.

[1] https://www.youtube.com/watch?v=FvmTSpJU-Xc [2] https://www.dartlang.org [3] Or in Dart lingo, a VM snapshot.

tracker15• 10 years ago

I'm fine with that.. however, I've seen MANY instances of websites that load multiple versions of jQuery... that's just one library. Let alone how poorly constructed most web applications are.

When it comes down to it, for so long many front-end guys only cared about how it looked, and backend guys didn't care at all, because it's just the ui, not the real code.

We're finally at a point where real front end development is starting to matter. I honestly haven't seen this much before about 3-5 years ago... which coincides with node and npm taking over a lot of mindshare. There's still a lot of bad, but as the graph show, there's room to make it better.

MustardTiger2• 10 years ago

Web browsers have more or less become mini operating systems

I wish. No, web browsers have become massively bloated operating systems. And since they didn't intend to, they are terrible at it. You have little to no control over anything.

stepanhruda• 10 years ago

Isn't WebAssembly a big step towards this?

tmd83• 10 years ago

I also agree that the web is full of bloat and a mishmash of compromised features. I think you can design a web thats more powerful yet simpler if you actually solve the problems in a more forward thinking manner.

For example there has been so many things for powerful layouts whereas everyone knew 10 years ago we need powerful layout solutions (flexbox or whatver) and now we have grid frameworks and years of craft on older css enhancement that has to be supported. They keep adding features here and there to sort of address lots of problems where individually those features might be cheaper but the overall cost of implementing both by browsers and the us regular developers is much higher.

Here are couple of the things I want from the web and quite a few of them are there already if not in super ideal forms. Powerful layout thats simple enough to use, concept of webpage, a bundle (http2? all your resource together), making the whole partial rendering (ajaxified page) a natural concept. Even making the UI/markup delivery being made separate from content (you can do that with all sorts of library but I think it should be at the core). Security concepts that are easier to implement (CSRF, url tampering etc.).

One of the idea I had is that browsers make a new engine that does the right things from the start and hopefully thats a much lighter engine and if you serve new pages they are really fast and if you serve old pages there is an optional transpoiler kind of thing that translates to the new version of the fly. Now it won't be terribly good to start with so its optional but essentially the old version is frozen and the more people start to only use the new engine (with transpoiler).

Maakuth• 10 years ago

Thanks to smartphones, people are already familiar with the modern concept of the standalone app; why not just make downloading an OS-native binary as easy as typing in a web address, on every OS?

Congratulations, you've invented ActiveX.

anthk• 10 years ago

Say if I press Cmd+Space on a Mac and type "Facebook" in Spotlight, it immediately begins downloading the native OS X Facebook app

Epic malware vector.

dayjah• 10 years ago

This whole thread is really interesting but I think it misses a key point: developing countries. The issue with these pages getting larger is that it creates two webs: one that work on the slower networks of less developed countries and another that works for the richer world.

It's often suggested that we'll solve poverty by providing education and given the internet provides a unique opportunity to learn then surely we benefit as a species by ensuring there isn't a rich web and a poor web? This risk is compounded by rich people being in a better position to create better learning opportunities.

Much like there's a drive toward being aware of the accessibility of your website (colour blind-check tools, facebooks image analysis for alt text tags, etc, etc) we should be thinking about delivery into slower networks in poorer countries.

morgante2• 10 years ago

You don't even need to do a complicated extension mechanism.

Users already respond to page load time. There's extensive evidence to support this.

niutech11• 10 years ago

In order to load much lighter web pages, just use Opera Mini or http://gdriv.es/l/ [append url]

hammock• 10 years ago

Can anyone explain why a simple web page is so much bigger now than a whole game?

robotnoises3• 10 years ago

Because we can! Games were slim back then because they had to be. Webpages are fat now because our computers are fast.

Now, if a webpage were to approach the size of a contemporary game...

degif• 10 years ago

A simple web page nowadays is bigger than a whole game in 1993. A whole budget game nowadays is several gigabytes in size and that is far from a simple website.

shawabawa33• 10 years ago

More because that "whole game" was incredibly small, not because a web page is particularly big.

A single good quality jpeg is around 1MB. If you have 2 nice pictures on your web page, suddenly it's bigger than doom.

ufo• 10 years ago

The biggest culprits are the layers of Javascript for and and tracking/analytics and unoptimized, high-resolution images and videos...

http://idlewords.com/talks/website_obesity.htm

rhinoceraptor• 10 years ago

For one, Javascript is used as source code text, whereas games benefit from being compiled to binary form. There's also a monetary cost for file size if your game requires two floppy disks rather than one.

cm32• 10 years ago

Because modern web sites circumvent the HTML engine and request JSON from the server which they then render via JavaScript.

Other bugs with the modern web authoring style:

1. broken back/forward functionality

2. broken scroll bars

3. broken accessibility

4. broken memory of where the page was at when closed, leading to a fresh load of the page on Re-Open-Closed-Tab

5. infinite scroll being highly unnatural

dreamlayers• 10 years ago

In the late 00s I remember turning on an old computer with a 650 MHz Athlon CPU and being surprised that web browsing performance in Firefox wasn't bad. Now if I try that with a 1 GHz Pentium 3, performance is absolutely horrible. Is this why?

fredley• 10 years ago

This, and Javascript, and browser bloat.

zokier• 10 years ago

Unless the browser ends up swapping a lot I doubt that browser bloat would cause significant perceptible performance regressions compared to the quite significant amount of optimization that has happened in the meanwhile.

toyg2• 10 years ago

Basically -- ultimately, it's the rise of javascript for third-party content (ads, trackers, APIs, add-ons, etc etc) that drives both page bloat and poor rendering performance.

JoeAltmaier2• 10 years ago

...and my first computer had 128 bytes of RAM. And a 300-baud modem.

NoGravitas2• 10 years ago

That's amazing, what was it? My first computer had 5K of RAM.

JoeAltmaier2• 10 years ago

Altair 8800, kit from MITS. And I wrote a game on it, using keyboard and screen.

jokoon4• 10 years ago

I just watched https://www.youtube.com/watch?v=Q4dYwEyjZcY this video about the early HTML standardization process, and it seems to explain all the ills of HTML.

So indeed, there is a huge optimization opportunity of having a stricter error model.

Also, I'm really wondering how much battery could be saved when surfing such pages.

Also I'm sure there is a lot of potential going in the pre-parsed document model. But that's a next level kind of engineering I guess.

robotnoises3• 10 years ago

Before everyone jumps onto the JQuery/Bootstrap/etc sucks bandwagon, just a reminder that the minified jquery from cdnjs is 84.1kb. Bootstrap is 43.1kb.

If you want your page to load fast, the overall "size" of the page shouldn't be at the top of your list of concerns. Try reducing the # of requests, first. Combine and minify your javascript, use image sprites, etc.

madawan• 10 years ago

or use HTTP2 which should make the number of requests largely irrelevant.

acdha• 10 years ago

HTTP/2 helps with that but the total size still matters. This is particularly relevant for resources like CSS which block rendering – even with HTTP/2 making it less important whether that's one big resource or a dozen small ones, the page won't render until it's all been transferred.

https://github.com/filamentgroup/loadCSS#recommended-usage-p... has a rather nice way to load CSS asynchronously in browsers which support rel=preload.

spiderfarmer4• 10 years ago

I see lots of websites that don't even cache static resources and use > 10 scripts and stylesheets, from a lot of different domains. Terrible.

eterm• 10 years ago

And link the non-minified versions.

(but the different domains actually helps speed up loading with http because it helps parallelise transfers.)

r1ch2• 10 years ago

That's still 84 KB of highly compressed javascript code to parse and execute. Even on a 4.4 GHz CPU core and a cached request, jquery takes upwards of 22ms to parse - that's just parsing - not even executing anything! Now add a bunch of other frameworks and utility scripts and your 100ms budget for feeling "fast" is gone before you even reach the HTML content.

robotnoises3• 10 years ago

That's a fair point, though my threshold for fast is probably a bit higher than 100ms. That might be a conditioned thing, though.

dclowd99013• 10 years ago

How long does it take you to start Word versus loading Google docs?

Has everyone gone insane? The web is absolutely incredible and JavaScript is absolutely killing it as a portable application language. Remember the dark days of flash? Stop complaining.

tedmiston3• 10 years ago

Not to mention that using the latest stable jQuery/Bootstrap from CDN means it's likely to be cached before a user visits your site.

aorth• 10 years ago

The top ten sites are significantly lighter than the rest (worth noting if you want to be a top website).

Wow. That's nice to see actually.

faitswulff• 10 years ago

I suspect that the top ten sites are lighter because they must perform under load rather than being a top ten site because they load quickly, but it is nice to see. Imagine the billions of man hours that would be wasted by humanity waiting for the top ten sites to load.

Tenhundfeld3• 10 years ago

Interesting comparison, if a bit arbitrary. It raises a couple of questions though.

1) How do the numbers come out when you exclude images?

It's valid and good to know the total sizes, including images, but that can hide huge discrepancies in the experienced performance of a site.

For example, a page with 150KB of HTML/CSS/JS and a single 2.1MB hero image can feel very different from a page with 2MB of HTML/CSS/JS and a few 50KB images.

If we're just interested in total bandwidth consumption, then sure, total size is a good metric. If we're interested in how a user experiences the web, there's a lot of variability and nuance buried in that.

2) What device and methodology were used to take the measurements?

In this age of responsive design, CSS media queries, and infinite scrolling/deferred loading, it really matters how you measure and what you use to measure.

For example, if I load a page on my large retina screen and scroll to the bottom, many sites will send far more data than if I load them on my phone and don't scroll past the fold.

I only skimmed the article and didn't dig in to the references. These questions may be answered elsewhere.

spiderfarmer4• 10 years ago

The speed index was invented for benchmarking this: https://sites.google.com/a/webpagetest.org/docs/using-webpag...

Tenhundfeld3• 10 years ago

Ah, very cool, thanks for sharing. I'm familiar with a lot of the tools for evaluating the performance of a single site (e.g., that I'm developing), but I'm pretty ignorant of the standard approaches for these types of larger scale benchmarking projects.

imaginenore• 10 years ago

For the last project I built the initial page load with the absolutely minimal JS that was embedded into the page. Then it loaded the rest whenever it needed it. My coworkers were shocked how quickly the page loaded.

It's actually better to show the user some progress bar, than the standard browser's "Waiting for yoursite.com".

You can get away with a lot without jQuery, while still having clean-ish code.

thom• 10 years ago

There's a reason that the economics of web development mostly work and the economics of games development mostly do not.

thedevil• 10 years ago

This probably requires more explanation.

stegosaurus2• 10 years ago

If web bloat is a problem, I don't think that looking at whether <insert buzzword framework of CURRENT_YEAR> can be removed is the answer.

I suggest that at the moment, we have basically two camps of website, with rough, fuzzy boundaries.

1. A place where someone sticks up an insight, or posts a wiki page, or whatever, to share some thought to others (if anyone actually cares). The blogs of many users of HN. Hacker News itself. Wikipedia. The Arch Linux Wiki. lwn.net. Etc. The sites are very roughly concerned with 'this is what I care about, if you do, great, this is useful to you'.

2. Commercial web sites that employ sophisticated means to try and enlarge market share and retain users. AB testing. 'Seamless' experiences which are aimed at getting more views, with user experience as an afterthought (a sort of evolutionary pressure, but not the only one).

Complaining that camp #2 exists is strange. It's a bit like lamenting the fact that chocolate bars aren't just chocolate bars, they have flashy wrappers, clever ingredients, optimized sugar ratio, crunchy bit and non crunchy bit, etc.

It works! A snickers bar is a global blockbuster, and 'Tesco chocolate bar' is the functional chocolate bar that just does the job, but will never attain that level of commercial success, it serves a different role.

-----

My personal view:

Fundamentally what I want when we click a link from an aggregator, is an 'article.txt' with perhaps a relevant image or two. Something like http://motherfuckingwebsite.com/ maybe.

But if a site actually does that, a website like The Guardian, I'd fire up wget, strip all the advertising, strip the fact it's even The Guardian, and read it like a book. If everyone does it then no-one makes any money, site dies.

So what we actually have is this constant DRM-style race to try and fight for our brains to get us to look at adverts. It's not about jQuery, it's about advertising, branding, 'self vs other' (the integrity of a company as a coherent thing), etc.

I don't know what the answer is here. I think this is why I find concepts like UBI so appealing - I find it kind of alarming that we seem doomed to infect more and more of the commons with commercialization because we haven't found a solution to keep each other alive otherwise.

oldmanjay• 10 years ago

With stories like these, you have to know where to find the real problem. At least some percentage can be ignored whenever nerds use the word "bloat" because it's a code word for "the only things that should be allowed are things I approve" in a surprisingly large number of contexts.

RodericDay• 10 years ago

I dig that you tie this domain-specific problem with that much larger social issue.

stegosaurus2• 10 years ago

Thanks. I've been struggling a lot with life in general lately (depression, etc), and it's nice to have some glimpse that someone else shares the view, maybe.

I think we have a real issue at the moment with people funneling their efforts into ways of dealing with issues at the micro level, rather than the macro level. Local optima. Something like that.

It makes me happy to see the sort of political revolutions that the Internet can bring about, but sad to see that we're still thinking so small. e.g. "Get x into tech" rather than "Make it so that you don't need to be in tech to live a decent life". That sort of thing.

forgotpwtomain• 10 years ago

How about browser bloat? Each chromium tab on linux takes an extra ~50-150mb depending on the site -- and I still have no idea what they need all of that memory for...

r1ch2• 10 years ago

Mostly Javascript and caches.

niutech11• 10 years ago

Chromium is a memory hog, it is a well known fact. Use Otter Browser or QupZilla instead.

Jerry2• 10 years ago

Maciej Cegłowski has a great talk/writeup on this very problem:

The Website Obesity Crisis

http://idlewords.com/talks/website_obesity.htm

Here’s the video of the talk if you prefer to hear him speak: https://vimeo.com/147806338

datalist2• 10 years ago

Not too long ago Medium pushed an "invisible" 1MB image to clients

https://binarypassion.net/digital-decadence-6ea59251d64d

and the video it refers to https://vimeo.com/147806338

damptowel• 10 years ago

At my frst job I once had to fix a wordpress site that was 'too slow' which loaded a 12MB 4k by 4k favicon. I kid you not. It also loaded jquery 4 times, had about 20 unminified stylesheets linked in the header. When I told this to my manager he told me 'oh but you only need to load that stuff once and then its cached, so that can't be an issue'. I did not last long in that place. The backend was even more of a wasteland.

datalist2• 10 years ago

A 16 megapixel favicon does sound scary.

donkeyd• 10 years ago

Visited a website a few days ago, which used 2048x1365 jpegs for 190x125 buttons. They had multiple buttons like this on multiple pages. I sent them an e-mail about this, but I don't expect them to fix it.

K0nserv4• 10 years ago

Quite happy with my own web page/blog. Pages hover at around 10kb, 30kb if I include some images. I think the page size can be attributed a lot to there being no JS except for GA.

I have taken a lot of inspiration from http://motherfuckingwebsite.com/ and http://bettermotherfuckingwebsite.com/

Of course the size will differ depending on the site's purpose, but I feel like most web pages could stand to loose a lot of weight.

EDIT: I have a guide to setup a similar blog/site here[0]

0: https://hugotunius.se/2016/01/10/the-one-cent-blog.html

gonzo41• 10 years ago

Plus one for you. It used to be the internet was a book, now it's a movie.

gkya• 10 years ago

I think the case of WWW and Internet is one of the biggest collective misunderstandings and tragic fails in the history of technology (and maybe humanity). If there would be a study of average ratio of information in kilobytes to page size in kilobytes, the result would make us cry. I have a Notes.org file that holds most of my notes, a list of most movies I watched and I'll watch, most books I read and I'll read, most places I saw and I'll see... basically my life and its about 200kb, where most probably more nearly 100% of it is information, whereas most multi-meg webpages have two paragraphs worth of information on them.

alasano2• 10 years ago

http://i.imgur.com/wB5B35g.png

My user experience was great, I have to agree.

gradi3nt• 10 years ago

"You're a fucking moron if you use default browser styles." - Eleanor Roosevelt

chriswarbo5• 10 years ago

I find it hard to take advice from anyone who makes such strong claims, in such a provocative manner, whilst failing so utterly in their own demonstration: https://imagebin.ca/v/2eeSxQABPMX9

Apparently that's "legible" and "looks the same in all... browsers".

I certainly agree that BMFW has less contrast than the MFW; so little that it's difficult to read!

For comparison, here's how MFW looks: https://imagebin.ca/v/2eeVa5QksF6e

White text on a black background, exactly as I asked for when I configured Firefox. Content taking up my whole widescreen monitor, exactly as I asked for when I made the Firefox window that size.

The only thing BMFW seems to have correct is using sans-serif font; which I expect is because I unticked Firefox's "allow pages to override these fonts" option. Other than that, it looks like a me-too cargo-cult of MFW which completely misses the point.

Presumably the creators of BMFW are using, and only ever test anything with, a black-on-white style which is, let me guess, the browser's default? What would that make them, in their own (jokingly "quoted") words?

jswny• 10 years ago

Great writeup! I might try this out later and see how it fairs. I haven't used the Jekyll + GitHub combo yet but I have heard great things.

K0nserv4• 10 years ago

Thanks, glad you liked it :) If you do give it a try and run into problems tweet me at @k0nserv.

nickpsecurity4• 10 years ago

That loaded SOOO fast on mobile with no delays or scrolling problems. Great job!

Note: Might try it with NoScript later. Always a good indicator of... something...

K0nserv4• 10 years ago

Fun story about NoScript. I used to use AMP[0]. But I reverted that for two reasons.

1. The size of their js is ~170kb roughly 17 times larger than most of my pages.

2. It loads from a 3d party domain which means that NoScript/uBlock Origin users will see a blank white page.

The project does impose some constraints that will help reduce page weight and increase page speed, but for my page sizes it's ridiculous to use their JS.

0: https://www.ampproject.org/

Malcx• 10 years ago

Shameless self promotion:

https://github.com/Malcx/DamnSmallBlog

BafS• 10 years ago

I have begun a 1 ko framework based on this website, if anyone is interested, I just pushed the code right now https://github.com/BafS/mu

niutech11• 10 years ago

There has long been http://www.1kbgrid.com

webscalist• 10 years ago

What happened to semantic markups? In the name of rendering optimizations, many web sites use css background image instead of <img>

AdamN• 10 years ago

The only real solution is a search engine that allows the end user to clip the results based on the maximum size of the total page. I've often wondered why Duck Duck Go doesn't do this as well as filter search results based on number of ad networks used, etc...

jordigh2• 10 years ago

Just to clarify, since I was confused (I remembered that Doom 2 was about 30 megs uncompressed, which websites are still a long ways from), this metric appears to refer to the compressed size of the Doom 1 shareware distribution.

http://www.doomarchive.com/ListFiles.asp?FolderId=216&Conten...

alasano2• 10 years ago

I'm genuinely excited by ensuring great response times and minimal load on a website.

Locally I see so many companies building good looking but horrendously optimized websites for their clientele who don't know enough to ask for it.

The last company I worked at were building a local search engine and were displaying thumbnails whilst loading full size pictures which were hot linked from businesses websites. With an auto loading feature at the bottom of the page by the php backend, an initial 5-6 Mb page load could turn into 30+ Mb within a few seconds of scrolling. Add to this no gzipping and caching was not properly configured either.

I tried my best to get some changes going but the senior (and only other) dev wouldn't allow any modifications to the current system "for the moment". It was a bit frustrating to see so many easy fixes ignored.

snowwrestler4• 10 years ago

The Doom install image was 35x the size of the Apollo guidance computer.

Thirty-five times! Apollo software got us to the moon. Doom wasted millions of man-hours on a video game.

My point of course is that these comparisons are not actually that illuminating.

Are web pages much heavier than they need to be? Yes. This presentation very capably talks about that problem:

http://idlewords.com/talks/website_obesity.htm

Does comparing web pages to Doom help understand or improve the situation? No, not any more than comparing Doom to Apollo memory size helps us understand the difference between a video game and a history-altering exploration.

skoczymroczny• 10 years ago

Apollo software got us to the moon. Doom wasted millions of man-hours on a video game.

Well, to be honest, Episode 1 and Episode 2 of Doom takes place on Phobos and Deimos, so you could say Apollo software got us to the moon but Doom got us to Mars :)

jordigh2• 10 years ago

Since 2.1 megs is only the compressed size of the shareware distribution[1], we are not going any further than Phobos. Since this is only for v1.0, it is going to be a buggy Phobos.

[1] http://www.doomarchive.com/ListFiles.asp?FolderId=216&Conten...

StavrosK3• 10 years ago

That link inspires me to print and bind the code of a webpage into a book, and put it on a shelf next to other literature masterpieces.

One Hundred Years of Solitude

The Count of Monte Cristo

Anna Karenina

Don Quixote

Amazon.com: Online Shopping for Electronics, Apparel, Computers, Books, DVDs & more

ArkyBeagle8• 10 years ago

I want the audio version, narrated by Malcolm McDowell.

tdsamardzhiev• 10 years ago

Remember: It's not a waste if you (and millions of other people) have fun!

matt4077• 10 years ago

That's what NASA wants you to believe...

Eerie2• 10 years ago

Apollo software got us to the moon. Doom wasted millions of man-hours on a video game.

How many millions of man-hours Apollo project wasted for a PR stunt?

manaskarekar2• 10 years ago

Mandatory: Benefits from Apollo [PDF] https://www.nasa.gov/sites/default/files/80660main_ApolloFS....

komali2• 10 years ago

Even if you could argue that Apollo was nothing more than a "PR Stunt", what is the issue with it being that? From a nationalism perspective, there's a tremendous advantage to be gained from plugging an entire generation with patriotism from a moon landing.

pessimizer2• 10 years ago

Doom was delivering an experience far more complex than the Apollo guidance computer was. The average webpage is not. It's delivering an experience as complex as a pamphlet with a few phone numbers on it.

pink_dinner2• 10 years ago

The experience may not be complex, but the software that builds and displays the experience is way more complex than the Apollo guidance computer.

harryf• 10 years ago

Are web pages much heavier than they need to be? Yes

What about the question "do web pages work any better than they did in 2007?" when we were using full page reloads and server side logic instead of Javascipt tricks.

I see so much basic brokenness on the web today from the back button not working to horribly overloaded news websites with mystery-meat mobile navigation I find myself wondering what have we really achieved in the last 9 years? This stuff used to work

untog• 10 years ago

I think you are looking at the past through incredibly rose-tinted glasses. The web has been a mess for a long time, and we used to have to make sure our computer was set to 800x600 and we were using Internet Explorer 6 in order to even use it.

collinmanderson• 10 years ago

This website itself is not far behind, weighing in at 937kb and 57 requests: http://www.webpagetest.org/result/160422_KJ_18KN/1/details/

jordache• 10 years ago

network latency still fails to catch up to human awareness' time resolution.

client-side logic, done right is much improved over a server-side solution.

falcolas• 10 years ago

Doom was created for entertainment. Web pages are (typically) created for entertainment. The guidance computer was created for calculating trajectories.

IMO, Doom and Web pages are remarkably close in terms of purpose and required assets, and the comparison is apt. Especially when you can play Doom on a web page...

PaulHoule• 10 years ago

Apollo era space navigation is not that complex, mainly a matter of (i) pointing the ship in the right direction, (ii) firing the engines until a certain velocity change had happened, and (iii) assessing the result. (ii) in particular is a one-dimensional problem, and (iii) can be done by the guys on the ground via radar.

What the shuttle did was much more complex because it was an unstable aircraft that required many "frames per second" applied to the control surfaces to keep it stable during reentry and landing.

Back around 2006 or so I wrote a simple software 3-d rendering engine in Javascript that was 8k in size without much effort towards minimizing size other being (a) maybe the only AJAX application that actually used XML (to represent 3d models) and (b) using XML element and attribute names that were just one character long.

Not long after that, libraries such as Prototype and JQuery were becoming popular and these were all many times bigger than my 3d engine before you even started coding the app.

treebeard901• 10 years ago

To be fair, Doom wasn't a waste by any metric. Doom has had an history altering impact on 3D engines. The huge advancements that came from the Doom and Quake engines have found their way into software that benefits society as a whole.

blazespin• 10 years ago

Pretty sure he was just being funny about the doom bit.

ArkyBeagle8• 10 years ago

The Web is heavy because there's no negative feedback for the weight factor. And in people's minds, it nearly seems that the difference between a video game and history-altering exploration diminishes day by day.

wldcordeiro• 10 years ago

Honestly if a native Android or iOS app can be several Gb in size then a webapp can be a few Mb. That said there are lots of optimizations that we're missing out on. I hope http2 and other advances in server side render + tree shaking help reduce the size of payloads further.

tompsonson• 10 years ago

The Doom install image was 35x the size of the Apollo guidance computer.

Keep in mind that the AGC was a necessary but not sufficient piece of hardware for navigating to the moon, and was extremely special-purpose. NASA had several big (for the time) mainframes that

1) calculated the approximation tables that were stored in AGC ROM (each mission required a new table because the relative positions of the earth, sun and moon was different)

2) reduced soundings from earth-based radars to periodically update the AGC's concept of its position.

3) other things that I've forgotten

In other words, the AGC required the assistance of a ground-based computer with dozens of megabytes of RAM and hundreds of megabytes of storage. That will fit on your phone quite easily, but let's not minimize the requirements for celestial navigation.

jccalhoun2• 10 years ago

that's because the moon landing was staged! wake up sheeple!!!!!!

TeMPOraL11• 10 years ago

Yes, it was recorded in a sound stage on Mars.

flexterra• 10 years ago

what is the average size of a native mobile app?

spriggan32• 10 years ago

The average data plan here is 10GB :

1,000,000 * 10 / 2250 = 4444 web pages a month

4444 / 31 = 143 web pages a day at most on mobile.

While it is somehow acceptable, I don't see data plans getting cheaper yet the size of the average webpage is raising fast.

It doesn't seem like most websites have heavily invested in using HTML5 offline capabilities or actual mobile first design either, something easy to check with chrome dev tools.

Also let's talk about ads : Polygon.com a site I visit often , first article on the homepage with an Iphone 5 :

- with ads/trackers 1.5mb - without ads 623kb

More than half of the load is ad/tracking related. This isn't normal.

intrasight• 10 years ago

The publishers really have no incentive to address this until a critical mass of users install adblock software.

elcapitan• 10 years ago

Doom would have been the superior solution to the World Wide Web.

njharman2• 10 years ago

So, were can I play Doom in the browser?

Kristine1975• 10 years ago

Here: http://playdosgamesonline.com/doom-ii-hell-on-earth.html (yes, it's Doom 2, and yes it requires a fast machine since it emulates a DOS PC in the browser)

nickpsecurity4• 10 years ago

I can't help but add that it's a DOS PC + a game that still loads faster than some web pages. At some point, even the apples to oranges comparisons start saying something because it's glaringly obvious.

njharman2• 10 years ago

Ok, awesome. Now I'm wondering if we PCDOS in a broswer...

1) Can we say the claim (I think of mozilla's) that the web was going to be the new OS is true?

2) If DOS today, will it be 5 or 10 years before we get BSD or *nix in the browser?

skarap• 10 years ago

Looks like most of the discussion here is on network traffic.

Minifying JS and CSS, compression, CDNs and caching won't keep your browser from having to render all the stuff.

---

The stewardess on a new jet airliner:

- Ladies and gentlemen, welcome aboard of our new airplane. On the second deck you'll find a couple of bars and a restaurant. The golf course is on the third deck. You're also welcome to visit the swimming pool on the fourth deck. Now - ladies and gentlemen - please fasten your seatbelts. With all this sh*t we'll try to take off.

dempseye• 10 years ago

I once bought a pre-made landing page template with all kinds of whizz bang Javascript libraries built in. The demo page was 4 MB. In the time it took to strip all the trash out of the template I could have designed the page myself. I'll never do that again.

I wonder how much of the problem is due to bloated templates.

Kurtz79• 10 years ago

It is hardly surprising, considering that a single picture taken with an average smartphone is probably already surpassing that by quite a bit.

Times change, and 20 years in tech is equivalent to several geological ages.

If anything, it cannot really be underestimated how some developers were able to craft such compelling gaming experiences, with the limited resources available at the time.

My personal favorite as "most impressive game for its size":

https://en.wikipedia.org/wiki/Frontier:_Elite_II

CaptSpify5• 10 years ago

disclaimer: my own blog - https://blog.thekyel.com/?anchor=Why_I_Block_Scripts_and_Ads

I kept looking for a "minimal" blogging platform, but they all had too much bloat/JS/etc. I guess minimal means different things to different people. I ended up just writing my own. The biggest post I have is 7.41 KB.

I used to be interested in front-end design, but since it's the industry standard to use $latest_framework, instead of tried and proven practices, I've given up on that idea.

detaro6• 10 years ago

One extreme end of the scale ;)

I feel like investing a few hundred extra bytes on some styling might be worth it, a la http://bettermotherfuckingwebsite.com/ vs http://motherfuckingwebsite.com/

CaptSpify5• 10 years ago

I'm of the mindset that our browsers are built backwards. For "content-oriented" sites, we should present text, and give the users tools in their browsers to present them as the user sees fit. Instead, we expect the site to design it. Something like Firefox Reader View.

Some people may like bettermotherfuckingwebsite better, but I personally don.t

"presentation-oriented" sites are a different story, of course

sergiotapia• 10 years ago

Arguably sites have been increasing in size for one simple reason: It directly results in increased sales.

Everything is sales.

If cleaner, 'purer' sites made more money you bet the average web page would be 10kb.

It's all about what translates to more sales. As such, you won't ever see a return to more traditional websites. Look at Amazon with it's virtual dress models, heavy as hell, but they most certainly land more sales.

niutech11• 10 years ago

Quite opposite. It is proved that longer page loading time is less conversion.

Joof• 10 years ago

From this title; maybe hacker news needs a twitter?

Also: You can't use average page weight when you are just looking at the top ten. That downturn could represent a single website; all others could be increasing in size.

bb85• 10 years ago

The top ten sites are significantly lighter than the rest (worth noting if you want to be a top website)

Isn't that that the top websites have a lot more ressources available to improve asset management, cleanup and refactor?

tyingq3• 10 years ago

This set of benchmarks is often helpful:

http://www.dynatrace.com/en/benchmarks/united-states/

Particularly the "last mile" or "Chrome Homepage" tabs.

They cover the top websites, grouped into categories like "Retail", "Travel", "Media", etc.

The disparity across competitors is pretty stunning, with some websites getting close to 1 second to download/render a page, and others taking 6, 7, 8 , even 10 seconds. And these are all big, well known, companies.

And, for the most part, it all correlates very well to total page weight, and total number of artifacts on the page (js/css/images/etc). There are some exceptions, but it's a pretty strong correlation.

seagreen• 10 years ago

Oh God.

Every discussion about the web will continue to be a mess until we clarify what we're talking about.

Let's try rephrasing the title a couple times.

Rephrase 1: "The average size of a webapp is now the average size of a Doom install".

Response: Interesting, but not bad! Heck, some webapps are games. "The average size of a web game is now the average size of Doom" isn't a sentence that damns the web, it's a sentence that complements the web! (or would if it was true, and it might be for all I know)

Rephrase 2: "The average size of web document is now the average size of a Doom install".

Response: Well this sucks (or would if it was true -- still we don't know). Simple documents should be a few KB, not the size of a game.

Basically our terminology is shot to crap. Imagine if 19th century engineers used the same word for "hand crank" and "steam engine". "Hand crank prices are skyrocketing! What's causing this massive bloat!" Whelp, that could mean anything.

The best solution: web browsers should enforce a clear distinction between "web documents" and "web apps". These are two different things and should be treated separately. This won't happen though, which leaves us (the rest of the tech community) to explore other options . . .

nulltype• 10 years ago

Web apps/documents both look pretty good compared to modern games, those things are 100s of MB at least.

seanwilson• 10 years ago

Lots of people are focusing on excessive JavaScript and CSS but these combined are easily dwarfed by a single high quality image.

Try visiting Apple's website for example. I can't see how you can have a small page weight if your page includes several images that are meant to look good on high quality screens. You're not going to convince marketing and page designers to go with imageless pages.

Doom's original resolution was 320x200 = 64K pixels in 8-bit colour mode. Even an Apple Watch has 92K pixels and 24-bit colour (three times more space per pixel) now, and a 15" MacBook display shows 5.2M pixels. The space used for high quality images on newer displays is order of magnitudes higher to what Doom hardware had to show.

arohner• 10 years ago

Images can load progressively, so pure HTML + one large image absolutely can appear faster to the user. The JS and CSS to load a SPA won't. Page weight is a good rule of thumb, but it isn't the be all end all of a good experience.

spriggan32• 10 years ago

Try visiting Apple's website for example.

Indeed, right now on mobile the biggest asset on apple.com is a 1.7 MB picture.

http://images.apple.com/v/home/cm/images/heros/environment_e...

The total size of the webpage being 2.5 MB.

niutech11• 10 years ago

That's why anybody should optimize their images, using e.g. Kraken.io or converting to WebP.

NelsonMinar• 10 years ago

Is there a Chrome extenson that shows the size of a web page? There's a good one for page load time that I use, but I want kilobytes with and without cache.

sugarfactory• 10 years ago

Google developed SPDY, an efficient binary representation of HTTP messages. Maybe they will do the same thing but for HTML. It would be much more efficient if one could design a binary representation of HTML that can only express well-formed HTML.

person15• 10 years ago

https://en.wikipedia.org/wiki/Binary_XML

maerF0x0• 10 years ago

This all comes down to cost. It is much cheaper to have "bloat" than it is to pay devs to fix it. And customers find it much cheaper to deal with "bloat" than to find smaller alternatives. Sure the average webpage is bigger than doom, but the CPU in my phone is approximately 100x (times multicore too?) than the 486 that ran Doom.

Sure, if man hours were free, we could trim it all down to (my rough guess) about 1/10th the size. But at $100 or even $10 an hour its just not worth it. Pay the GBs to your carrier, spend $50 more on a better phone.

niutech11• 10 years ago

It would cost no extra time if devs made websites from scratch with performance in mind. Save optimized images, minify code as a build task, etc.

apeace• 10 years ago

This page clocks in at 935kb in my browser. According to this same page, that is roughly the size of Sim City 2000.

CM30• 10 years ago

It's also about 2 times bigger than a lot of SNES and Mega Drive games. Or about 4 times bigger than Super Mario World (512KB).

As for why it's getting so insane, probably either:

1. Frameworks, since most people don't remove the code they're not using. For Bootstrap or Foundation, that can be a lot of extra code.

2. Content Management Systems, since stuff like WordPress, Drupal, Joomla, any forum or social network script, tend to add a lot of extra code (more so if you've added plugins).

3. The aforementioned tracking codes, ads, etc.

chasing• 10 years ago

The only conclusion I can legitimately draw from this article is that in twenty years a single web page will be larger than the 65GB Grand Theft Auto V install.

perseusprime11• 10 years ago

Enable Ghostery and load cnn.com and you will see why the web pages are so heavy these days.

jakobdabo• 10 years ago

Yesterday I discovered that Twitter's HTTP headers alone are ~3500 bytes long (25 tweets!) with several long cookies, custom headers and the Content Security Policy[1] containing ~90 records. Is this considered normal nowadays?

[1] https://en.wikipedia.org/wiki/Content_Security_Policy

stepvhen• 10 years ago

Recall that Doom is a multi-level first person shooter that ships with an advanced 3D rendering engine and multiple levels, each comprised of maps, sprites and sound effects.

Doom isn't in true 3D, its an advanced raycasting engine. The levels are all 2D, there are no polygons, you can't look up and down. Doom has been ported to a TI Calculator. Lets maintain some perspective here.

coldtea• 10 years ago

Lets maintain some perspective here.

I see what you did here.

qaq• 10 years ago

Too bad we can not measure it football fields

kgr• 10 years ago

Send “models” rather than code. Low-level code is relatively unexpressive, contains considerable redundancy, and as a result, is relatively large. By sending high-level models instead, which are then expanded on the client to working code, application download size can be greatly decreased. Models typically provide one to two orders of magnitude of compression over code.

This video shows how we do it: https://www.youtube.com/watch?v=S4LbUv5FsGQ

This document gives some results (like a GMail client that is 100X smaller): https://docs.google.com/a/google.com/document/d/1Kuw6_sMCKE7...

damon_c• 10 years ago

In 20 years the average size of web pages will be the size of a Quake 3 install. This is progress.

dclowd99013• 10 years ago

Are we really complaining about webpage size when fully 30% of web traffic is Netflix? This might be an unpopular opinion but websites are no longer just html, css and js. They're full on applications with rich interaction and data visualization. Call me when they're larger than an average modern native app install.

Shivetya• 10 years ago

and here I remember that the PDF of the Turbo Pascal manual was so many multiples of the compiler's size I needed a calculator to figure it out

hackertux• 10 years ago

I also recommend https://news.ycombinator.com/item?id=10820445

jccalhoun2• 10 years ago

and the .kkreiger beta only uses 96k! https://en.wikipedia.org/wiki/.kkrieger

brownbat• 10 years ago

ronan has an account here, commented on this as it was developing a few months ago:

https://news.ycombinator.com/item?id=9981707

AndyKelley• 10 years ago

I wanted to see how one of my personal projects compared, so I looked at Groove Basin.

Groove Basin [1] is an open source music player server. It has a sophisticated web-based client with the ability to retag, label, create playlists, stream, browse the library, chat with other users, upload new music, and import music by URL.

I just checked the payload size of a cold load, and it's 68 KB.

I'll just keep doing my thing over here.

[1] https://github.com/andrewrk/groovebasin

awqrre• 10 years ago

It's initially cheaper to make larger web pages, you don't have to optimize for size (most of the time it would probably execute faster if it was smaller but probably not always). Some others make it larger on purpose for obfuscation (like Google).

partycoder• 10 years ago

For Internet 3 we should call John Carmack and put all of the internet in a MegaTexture.

myared• 10 years ago

GTA 5 is ~65GB in size. One day, web pages will be bigger than that.

LordKano• 10 years ago

Great, another kooky unit of measure.

"This new re-design gets us down to 0.4 Doom installs without sacrificing any of the visual elements."

tacone• 10 years ago

But they load faster than Doom used to.

systematical• 10 years ago

Amazon isn't what I'd call "light" at 4.7 MB, but looking at my companies market all the bigger players are way lighter than us.

ivanhoe• 10 years ago

Doom is not a good measure, when web pages become bigger and more bloated than your average printer or scanner driver, then it will be alarming :)

bendbro3• 10 years ago

Why can't all these frameworks just be cached? If a cross site request to cdn.com/react-v1.0.js is cached under cdn.com, at most one download will trigger. That seems to solve the problem, but maybe I'm missing something.

bendbro3• 10 years ago

I've seen people mention that caching is broken since it only works when the name and version are the same; however, I don't see how this is any different than two packages of the same functionality being uploaded to an APT repo with different names.

chowes• 10 years ago

Wondering if an idea like this would work:

Bundlers like Webpack already import JS in a modular structure. I'm wondering if we could do some profiling into popular npm module combinations (I know many people using React + Lodash + Redux Router, etc), bundle them up, and have Webpack load in those combos from a CDN via <script>?

Now this would probably require some work on webpack's end (the __webpack_require__(n) would have to be some sort of consistent hash), but at least everyone who blindly require('lodash') will see an improvement?

necessity• 10 years ago

Web pages shouldn't need JS at all, except for the basic eye candy it was originally used for, and most importantly shouldn't break without it. A web page that doesn't work without JS is broken.

Web apps are another deal though.

jrl• 10 years ago

This is one of the reasons why I love a simple website without too many whistles and bells.

Khaine2• 10 years ago

Isn't the answer to just create a reasonable standard library for javascript so people don't need to link in megabytes of frameworks

The average size of Web pages is now the average size of a Doom install

Related Stories