JavaScript Module Loaders Considered Harmful

http://techblog.ironfroggy.com/2014/02/javascript-module-loaders-considered.html

by nih • 12 years ago

58 42 12 years ago

techblog.ironfroggy.com

spellboots• 12 years ago

These complaints ring true for trivial javascript projects, however if you are building complex javascript heavy applications the equation quickly changes.

Point 1: You will almost always be transpiling your javascript. In any serious project, you will want to have lots of small files to ease development, and then concatenate into larger files to ensure optimal http loading. If you don't already do this regardless of modules, you probably should.

Point 2: If you are intentionally or serendipitously relying on script execution order, that is your bug. The module loader has helpfully exposed a bug in your code for you.

Point 3: If you have lots of modules, it is simpler to deal with a module system than mashing together a load of scripts.

This complaint should come with a severe disclaimer that it only applies to trivial applications, because as soon as you start building something complicated the arguments are just plain wrong.

Software engineering is hard, and robust software engineering practices usually only pay off once you get beyond trivial examples. The ones that work, however, pay off massively once you do use them for more complex use cases.

jared314• 12 years ago

For those devs, like myself, who are unfamiliar with large javascript projects, do you have a link to an open-source javascript-heavy project that fits your points?

jkrems2• 12 years ago

jQuery: https://github.com/jquery/jquery/blob/master/src/ajax.js

ACE: https://github.com/ajaxorg/ace/tree/master/lib/ace

wyuenho3• 12 years ago

You will almost always be transpiling your javascript. In any serious project, you will want to have lots of small files to ease development, and then concatenate into larger files to ensure optimal http loading. If you don't already do this regardless of modules, you probably should.

I think he was alluding to browserify and the many AST transformation preprocessors it inspired[1], many are just there to deal with other AST transformations....

You don't need a module loader or packager or whatever to concat or order your scripts, tools like Webassets or Sprockets can do that without any new JS code. Hell, a simple Makefile that calls `cat` does a far better job than most stuff out there.

If you are intentionally or serendipitously relying on script execution order, that is your bug. The module loader has helpfully exposed a bug in your code for you.

I call nonsense on this one. Scripts almost always have to execute in order. Can you run a jQuery plugin without having jQuery loaded and executed first?

If you have lots of modules, it is simpler to deal with a module system than mashing together a load of scripts.

It may be simpler to deal with files, but definitely not simpler to deal with JS module loaders. I suspect that the current proliferation of JS module loaders / packagers are due to the long time lack of tools on the PHP side to provide something like Sprockets and the fact that JS has no import/export mechanism built into the language. Tools like RequireJS/Browserify/Bower/Component are all abominations in their own way.

[1] https://github.com/substack/node-browserify/wiki/list-of-tra...

camus22• 12 years ago

You can blame the people in charge of the ES spec. that's the single most important thing to have in any language,how to properly import external files. Having to rely on the DOM to do that is a horrible hack. Requirejs is a DOM library,not a javascript library.

awinder2• 12 years ago

I think the question is where you want to say that the jQuery plugin should load before jQuery. If you have 20 jQuery plugins, which cross-leverage some of the same dependencies, and then all rely on jQuery -- do you want to rely on ordering script tags for dependencies in the HTML? Or do you want to leverage a module system that lets you coherently state what the dependencies are?

Also I'm not sure how Bower got dragged into this since it has nothing to do with module systems. It is a nice tool though for just tracking what third party libraries are in a project and their versions. Not sure what they did to prove to be an "abomination" :-)

inglor• 12 years ago

More like: point 1) source maps

seldo• 12 years ago

Agreed. On the front-end, module loaders eventually come in handy but their utility isn't immediately useful. If an "anti-pattern" is something that seems useful but eventually isn't, and a "pattern" is something that seems useful and immediately is, we need a phrase for this third state, where it's a pain in the ass to start with and only becomes useful later.

Chris_Newton• 12 years ago

If an "anti-pattern" is something that seems useful but eventually isn't, and a "pattern" is something that seems useful and immediately is, we need a phrase for this third state, where it's a pain in the ass to start with and only becomes useful later.

Technical investment, by analogy with technical debt?

jwarkentin• 12 years ago

I disagree that it's really even that hard to start with. It's like any library or framework. You have to learn it initially, but then it's just not that hard. I've been using RequireJS for a couple years as well as node.js and none of the complaints really resonated with me.

I would go so far as saying that the author has probably been using them wrong. If you follow the right patterns and write good code it should be a non-issue.

tieTYT• 12 years ago

Could you elaborate on Point 2? What causes this bug, and how do you fix it? I'm looking for a low level explanation rather than, "Use a Module Loader" because I want to fundamentally understand what's going on when my code depends on load order and how to fundamentally fix it.

masklinn• 12 years ago

Could you elaborate on Point 2? What causes this bug, and how do you fix it?

Unless @async is specified, the scripts linked in a page will be executed in that order. This means it's easy to unknowingly create implicit (and possibly unexpected) dependencies between scripts.

Because a module loader will perform asynchronous loading and execution of scripts, if dependencies are left unspecified the loading order is essentially random (mostly driven by the script's source size and how fast the server happens to respond to that request). A module loader requires that dependencies be explicitly spelled out, or changes are the application will randomly blow up at loading.

kybernetikos• 12 years ago

Unlike some languages, javascript files execute as they are loaded. In large projects relying on js files to take actions beyond the most basic definitional type things is usually a mistake because it means that there is evolving global state (always difficult to keep track of) with implicit semantics. It's very distressing to move some code around in the include order for an unrelated reason and then suddenly find that hundreds of unit tests start breaking with bizarre errors.

The way to avoid this is to limit your js files to definitions, and then finally kick everything off with a single call. Module loaders enforce this, but you can follow this advice without them.

It's reasonable to have one file that must be loaded first - this provides the mechanisms you're going to use to define your objects and nothing more. Then everything else can be done in whatever order seems good, before finally kicking off the action. Even registering code with factories and registries should be left until the end. There are some ordering requirements that are legitimate - if you inherit from another class, then the super class will typically need to be ordered before the subclass.

If you're doing a small project where you don't mind having up to 10 script tags in a page, then you'll do better following the 'separate definition from execution' advice than not. You'll do better still using some form of module loader that makes concerns of order obsolete and will let you scale. I like browserify, but there are lots of options, ranging from the very simple to the complex.

agilebyte• 12 years ago

I'll bite, is this article supposed to be serious?

Ad #1, maybe my projects have been too small, but I never felt lost even when using CoffeeScript as the source. I mean I can see in the comment above the module which file it relates to.

Ad #2, not sure this applies to CommonJS.

Ad #3, there are really just 2 standards. Not sure that just saying what the heck, I'll just use the global `window` object is a better solution when trying to integrate multiple different vendor libraries and your code.

svachalek• 12 years ago

Load order is a lot of what modules are there for. If A depends on B, B needs to load first. If you are depending on side effects of the module load or other undocumented dependencies, you should Google "considered harmful" and spend a few days reading the results.

All of the issues in #1 and #3 don't actually come from modules, but from "doing it wrong" -- preprocessors and forced dependencies and all the other bad practices that have been tacked on top.

awinder2• 12 years ago

To the point of module loaders making it difficult to understand load order, I'm not sure I understand the problem. The load order is pretty much trivial since I'm loading modules that don't execute on load. They execute on the order that I've defined, so all should be expected there. When you need an expected load order as when you're defining scripts in the document as synchronous loads, you're defining implicit dependencies. One of the great strengths of AMD systems like requirejs is that it turns implicit dependencies into explicit ones.

TimFogarty• 12 years ago

What are the supposed benefits of using module loaders over say minifying and concatenating all your JS into one maybe two files which are then placed in script tags at the bottom of your HTML?

jkrems2• 12 years ago

- Organized code (into modules)

- Explicit dependencies (instead of using the global scope for communication)

- Easier to reason about and test parts of the code

- Errors are properly isolated in a module

The post is putting module loaders (requirejs) and "using modules" (e.g. commonjs) into the same box. With browserify you end up with one or maybe two large concatenated file that you then place in script tags at the bottom of your HTML. There's no "instead".

loganfsmyth• 12 years ago

There is a distinction to be made between having a module system, and how those modules are loaded. You can still concatenate everything together while using a module loading system, you just don't need to load the modules individually in that case.

One potential problem with concatenating everything without modules is that you then need to maintain your concatenation order based on dependencies using some other system. To his second point, you can certainly do this perfectly easily on smaller apps, but it becomes impossible as systems grow.

snird• 12 years ago

Modules are evil in my experience, they have many disadvantages, much more than the writer points to.

When I looked up for some wrapper to Backbone I came across ChaplinJS: https://github.com/chaplinjs/chaplin/ and it was nice, but using the module system was a nightmare. It was so painful, that I forked it and remove the module system from the code (which was a pain, but still less painful than handling the module system). the fork is here: https://github.com/snird/Mildred and I use it in production now.

nilgradisnik• 12 years ago

Can you elaborate on what kind nightmare/pain you came across using AMD? From my experience requirejs simply gives you a bit of structure, helps you organize your code so you can reference other parts of it easily.

GeneralMayhem• 12 years ago

Um... I don't know what module system this guy has been using, but I use Require and none of these complaints gave me any pause.

Point 1: Chrome Dev Tools has no trouble whatsoever with AMD. It remembers breakpoints across page-loads just fine. And there's no mismatch between what I see in my source and what I see in the browser, because I load up the un-minified source files for development. Seriously, if you're trying to debug compressed files, the problem is not your module system.

Point 2: This is an argument in favor of module systems.

Point 3: This is the only one that's even slightly valid, but it's only a problem when starting up a new project. Once your Gruntfile is in order, the workflow is simplified - I no longer have to put that extra script tag in, I simply add more files to source, tie everything together just like I would in a desktop app, and they magically appear in the browser.

hippich2• 12 years ago

When you JS codebase get bigger, it will benefit loading pieces of code when needed. I.e. instead of loading one giant chunk of minified javascript covering all possible states, you load some "global"/"universal" js file and .js file used to render current state. When state changes - another small .js file get loaded and launched.

While all of this can be done using global variables and inserting <script> tags when you want to load module and wait for callback to fire, or setInterval check for module to be loaded, all this already done for you in libraries like RequireJS.

I would say it this way - yes, on small projects you can get away without using more abstract approach to split your project in pieces, but having module-based framework put into project early will allow you to scale code when you application get bigger.

camus22• 12 years ago

when needed. I.e. instead of loading one giant chunk

IMHO , The problem is the user is waiting for the functionality to be loaded,not a great user experience especially on mobile,where connectivity can be bad."Bulk" loading is usually cheaper than multiple requests to load parts of an application.

I usually load all the code required upfront,even data is fetched and cached in the background so my users dont have to wait while browsing multiple pages of a listing , for instance. I would never use async module loading in production. It often fails according to my tests.

Maybe using amd during development makes sense on some projects but not in production.

hippich2• 12 years ago

I am glad that you app can be actually served as one packaged piece without download size having any effect on performance. In my case loading everything is slower then loading needed pieces when needed. I am not talking about loading each view separately, i am talking about splitting application into "sections" and load each one when context is switched.

And yes, what we do here does make sense in production.

mariusmg• 12 years ago

I hate them . JS modules literally break any tool you throw at them. And the gains in code "modularization" aren't really that significant. Namespaces in JS are decent.

jamesaguilar• 12 years ago

Somehow, "considered harmful" has become a strong signal that the arguments in the so titled article will be weak. Why is that?

mcantelon• 12 years ago

Maybe because "considered harmful" is an unimaginative title component and a lack of thought in the title might be indicative of a lack of thought in general.

mistercow• 12 years ago

However, the fact is that the vast majority of our projects don’t have such complicated intra-module dependencies to really justify this.

I feel like this sentence is revealing. The problem isn't with module loaders at all. The problem is with using the wrong tools for the job.

The author clearly simply does not work on the kind of projects for which module loaders are intended. If you are working on small projects with few dependencies, your debugging, loading, and workflow are going to be uglier for adding a module loader.

But the bar for when that situation reverses isn't very high. Even a medium-sized project can quickly reach the point where you have dependencies that are difficult to manage by hand.

And if you opt to do manual script tags in that case, you're in for headaches. Debugging improperly ordered scripts is a pain, understanding the required load order is difficult, and your workflow is bogged down when it comes to adding new libraries. It isn't as simple as adding a new script tag, because you have to figure out where to put it.

On the other hand, pretty much none of the gripes listed in this article matter if you use something like browserify.

guiomie• 12 years ago

I've never understood why I should use a module loaders ?

I simply declare my .js files in my main html page and the is it ...

christocracy• 12 years ago

This is unmanageable for complex, "single page apps composed of many files (eg many tens of files). Often a complex app might deal with role-based features, where only certain sets of controllers are included per role.

ianstormtaylor• 12 years ago

The author makes the mistake right at the very beginning of conflating module loaders and specifications for modules (like CommonJS).

Module loaders, that is code that loads other modules onto the page after page load, are overkill. Most sites just don't have enough JavaScript logic to make the size of their JavaScript (compared to the two huge images on their landing page) the biggest problem they should be worrying about. Just concatenating all of your modules into a single script and call it a day.

But module loaders have nothing to do with the module definitions themselves, like CommonJS. Separating code into small modules is useful for 90% of projects too, basically everything other than single page, or completely static sites. And if you're thinking that modules are overkill then you're probably using a poorly designed system for organizing them, that adds too much friction to the creation or installation process.

Unfortunately that confusion in the introduction is going to make this comments thread go all over the place, for what could have been a could argument against complex module loaders.

pjmlp• 12 years ago

Sorry but no.

Any language intended for large scale development has to support modules.

Until ES 6, this is what we have. I surely will keep on using them.

octatone2• 12 years ago

I would never do a large JS project with out modules. But I also never use "module loaders".

ttty• 12 years ago

Cross post from the site comments:

I really can't see the problem of harm #1. If you use require.js, by default the same javascript file you are editing will be used by the browser, not changed in any way. You don't even need source maps for this.

The order of the modules don't really matter. For example you have module A and in order to work properly requires "module B" and "C". You don't care when the modules A, B or C are loaded because when you use the module A, the module B and C are already available to be used (thanks to the module loader). For example:

// moduleA define(['moduleB', 'moduleB'], function(A, B){ // here you have module A and B // and I don't really care the real download order. // That's the job of the module loader })

In require.js you can also define the dependencies of third party libraries and which variable they export (check require.js config). For just 20 lines a library can work with require.js AMD modules and with Commonjs. This is nothing compared with the total lines of the libraries, when you take in account the benefits.

I never liked to have multiple script tags in my html files because when you need more than 20 it really get messy. You also have to take care about dependencies at a global level, which is harder.

From what are you saying you write some big javascript files that contains your whole application. I prefer to keep each class in their javascript file and keep it as organized as possible. The order of the loading of the modules is not up to me to think about.

I just want to define a module and it's dependencies. I also don't want to manually select the files I need to have in the build file (a file that contains all the small files). Require.js allows you to create a big javascript file for deployment with only the modules you required, not more.

It's just crazy to not use some kind of module loader today. Check my explanation on how to use require.js with yeoman (http://www.webdesignporto.com/3-steps-to-fully-automatized-j...). It's just a lot more easier to develop big web applications, if this is what you want. Of course if you only build some websites that requires jquery to show up some popups and this kind of stuff, is overkill to use a module loader.

Hope you might change your mind about module loaders.

mxfh• 12 years ago

Don't worry it only gets better soon with ECMAScript 6 Modules http://wiki.ecmascript.org/doku.php?id=harmony:modules

or does it? http://calvinmetcalf.com/image/75296462861

sdgfnlk• 12 years ago

I don't agree with his point #2 in two ways. For one of our internal applications, we had so much <script> tags embedded into our single-page-app index.html that it was very hard to keep track of dependencies. If you have a straight dependency order, sure it won't matter, but if you start refactoring things out and one module has a bunch of dependencies that also has dependencies, this essentially degrades into rolling out a manual topological sort of the javascript files you include. On top of this more than one engineer works on the application, and different engineers inserting <script> tags at the same time caused a few headaches.

To mitigate this, we introduced RequireJS to our internal app, and now the dependency is declared at module definition so you don't have to sift through a list of <script> tags. You don't fear breaking dependencies of each script files because you put one on top of the other. IMHO it is much better to declare the dependencies to your module explicitly than implicitly through a very carefully topologically sorted series of <script> tags.

rafekett• 12 years ago

https://github.com/spine/hem

JavaScript Module Loaders Considered Harmful

Related Stories