Keep Webpack Fast: A Field Guide for Better Build Performance

https://slack.engineering/keep-webpack-fast-a-field-guide-for-better-build-performance-f56a5995e8f1

by firloop • 9 years ago

214 47 9 years ago

acemarke• 9 years ago

Excellent summary of some of the techniques you can use to speed up Webpack builds. It'll be interesting to see how the ongoing work on performance in Webpack 4 relates to this [0] [1], and I'd guess that the just-announced Rust->WASM sourcemap lib work [2] will help too.

If anyone's looking for more info on this topic, my React/Redux links list has a large section of additional posts on improving Webpack build perf, code splitting, bundle size optimization, and other related techniques [3].

[0] https://github.com/webpack/webpack/issues/6244

[1] https://twitter.com/TheLarkInn/status/945486181575340032

[2] https://twitter.com/TheLarkInn/status/954053251854344192

[3] https://github.com/markerikson/react-redux-links/blob/master...

simooooo• 9 years ago

I use webpack every day. But the config for it scares the crap out of me.

osteele• 9 years ago

I recently discover Poi https://poi.js.org/. It’s built on top of webpack. If your issues with webpack are configuration complexity, not speed, then Poi may be appealing to you.

edmorley• 9 years ago

Another option similar to Poi, that I'd recommend adding to any comparison, is https://neutrino.js.org/

shawn• 9 years ago

If anyone is tired of Webpack configuration issues / slow performance, give Parcel a try:

https://parceljs.org/

The project is less than two months old, so Parcel might not be a good fit for you. But if it is, it's usually a drop-in replacement. Zero configuration. Just run it and go.

We're working hard to add the most requested features: source map support is coming very soon, for example. But if you can put up with a few rough edges, Parcel promises to give you a ~5x performance increase over webpack.

(We kick off multiple processes to do the actual bundling work, and we have an asset cache, which accounts for most of the speedups.)

If you'd like to participate or have any questions, hop on our Slack! https://slack.parceljs.org/ We have about 500 devs ready to help you out and chat with you (about anything, really).

Feel free to snag an issue and start working on it: https://github.com/parcel-bundler/parcel/issues?utf8=%E2%9C%... There's plenty of work to go around.

It's been really cool to see so many people pitching in. We recently had a Japanese contributor help get Parcel up and running for multi-language projects: https://qiita.com/zacky1972/items/0ce05454b67506edc634

chickenfries3• 9 years ago

Is parcel the first node module bundler to run multiple cores? I’ve been waiting for this forever!

Definitely going to take a look at the code and try this out soon.

chmln• 9 years ago

I believe webpack can run using multiple threads at least [0]

[0] https://github.com/webpack-contrib/thread-loader

chrismorgan2• 9 years ago

There’s a reason why it’s not generally done: there are no good tools for doing this in Node, nor is JavaScript particularly conducive to it.

The state of the art in such matters is Rust with its type system-assisted fearless concurrency; my favourite demonstration of how it can work is Rayon, where introducing data parallelism is typically as simple as replacing a .iter() with a .par_iter(). I’d love it if something like Rayon could be ported to Node, somehow.

(By virtue of being excellent at the low-level and having zero-cost abstractions, Rust actually ends up being a really good high-level language.)

In Node, if you want to process a zillion files in the same way, there are no good solutions. I recently needed to improve the performance of a manually-crafted, Node-centred build process; in Rust, it would have been a matter of: add Rayon, replace a .iter() with .par_iter() and we’re done—and what’s more, it’ll only flex its parallelism muscles where it’s worthwhile. But in Node it took a several hours of trouble (after research, I settled on using worker-farm to help) to produce a strictly inferior result.

Rayon: https://github.com/rayon-rs/rayon worker-farm: https://www.npmjs.com/package/worker-farm

PudgePacket• 9 years ago

Maybe a naive question but did you consider writing it in language that is typically faster (C++/Go/Rust)? I understand the many benefits of it being written in JS but there is an inherent maximum level of performance that JS can achieve in certain workloads.

thangngoc892• 9 years ago

There is an attempt to write JS module bundler in OCaml https://github.com/fastpack/fastpack

fenomas• 9 years ago

My naive answer, but I'd think that the heavy lifting for bundlers like this lies mostly in the external packages that parse and modify the code (Babel, uglify, postcss, etc). The core work of the bundler to wrangle which dependencies to pass to those external libraries probably isn't a performance bottleneck, surely?

merb• 9 years ago

well at least there is a tool in java. and once warmed up it's basically the fastest tool available. google closure compiler. so basically if there would be a closure service that could be installed on your system a lot of things would be way faster.

JoshGlazebrook• 9 years ago

I tried out Parcel this past weekend and I was really impressed with how easy it was compared to when I tried using Webpack for the first time. Just import what you need and it find it all and you're good to go.

hazza1• 9 years ago

Parcel is a neat solution, lack of sourcemaps stops me from using it for anything serious but I believe this is being worked on.

Fiahil• 9 years ago

I would prefer to stick with webpack and make it easy and fast again, rather than changing tools -again. We've seen this with grunt, gulp, and now webpack. What if, we're changing strategy this time ?

purerandomness• 9 years ago

Nah, I heard Yeoman solves all these problems.

JimDabell2• 9 years ago

If anyone is tired of Webpack configuration issues / slow performance, give Parcel a try

Shawn, half the articles about Webpack I see recently have these kinds of comments about Parcel. This goes beyond excitement into spammy behaviour. I've seen plenty of other people complaining about this too.

This was an article about Webpack performance, and you've come along to talk about unrelated Parcel features like source map support and i18n, to link to the Parcel Slack, and to ask for help working on Parcel bugs.

It really puts me off Parcel and there's no chance I'll look at it so long as when an article about Webpack pops up, there's always a comment from a Parcel developer spamming your project. Please stop spamming Parcel in Webpack discussions.

kreetx• 9 years ago

But why not actually try it then, and come back with what you got? I.e maybe he's here because they solve most of the pressing issues.

(I myself have no skin in the game either way)

ksec• 9 years ago

I am sorry if this is silly question, and no offence intended.

Is Webpack beyond fixable? Tech has always been about reinventing the wheels, but the JS community seems to dial this up to a whole new level.

Isn't Webpack 4 heading towards Zero Configuration?

Isn't Webpack 4 making many performance improvements?

This isn't to say i dislike Parcel. I mean God I wish Rails could have picked up on it as default.

dbbk• 9 years ago

Out of interest, is there an overview of Webpack 4 somewhere? This sounds interesting and it's the first I've heard of it.

xab9• 9 years ago

This sounds great, but in reality most projects became so dependent on webpack, that moving to another packager would be a time waster (at least from managemen't spoint of view - and heck, I might agree: for us it would take weeks if not months).

Sacho• 9 years ago

Why is parcel x5 faster than webpack?

znt• 9 years ago

Maybe it's because you don't spend 5 hours reading through webpack documentation and examples?

jamesmcintyre• 9 years ago

From someone who has experienced the dev flow on apps with very large, gnarly webpack configs 17 seconds is extremely impressive and sounds like heaven!

What a lot of devs may not be aware of, if their projects aren't of a certain scale & complexity (support for legacy code/library-choices/extreme-modularity/etc.) is that the magical nearly-instant hot-module reloading you experience with a clean, "normal" project should not be taken for granted because webpack builds/reloading can become a monster! Not to say you can't scale up good webpack performance (obviously slack has) just that it's not an automatic thing but something that requires some TLC.

Thanks for taking the time to share slack team!

Game_Ender• 9 years ago

Can anyone explain how someone can make a build system and not make correctness and speed the top priorities from the start? Once you do that parallelism is third feature you implement, the first being running an action, and the second being a hash based dependency graph of those actions inputs & outputs. Once you have that dependency graph parallelism and caching just appear pretty much for free.

It looks like google uses bazel (a parallel dependency graph based build) internally and they have 1-2 second rebuild times on apps much larger then Slack [0].

0 - https://medium.com/@Jakeherringbone/what-angular-is-doing-wi...

swsieber• 9 years ago

Keep Webpack Fast: A Field Guide for Better Build Performance

Related Stories