Tiny Treeshaker: JavaScript tree shaking in 200 lines of code

https://github.com/magicmark/tiny-treeshaker

by markl42 • 5 years ago

62 54 5 years ago

github.com

Waterluvian• 5 years ago

Something that drives me bonkers about current tree shaking is that there’s no good way to see what got shaken.

I want a graph that shows me all the branches that got culled.

Even better would be a dependency graph that shows me what is causing things not to be culled.

I had an issue where Antd was pulling in 200KB of icons and I couldn’t figure out why until I spent hours bisecting it by commenting out entire sections of my application.

cookiengineer• 5 years ago

It would actually be quite amazing if dead code analysis of branches are integrated into eslint or other linters.

But yeah, I guess that would also require a VM like understanding of timeouts, intervals and the Browser's triggered events so it's kinda hard to do.

mirekrusin2• 5 years ago

It doesn't have to be perfect, from static analsis you can get what people want. It won't cover dynamically created keys but people don't use it anyway. You'd just have to provide list of entrypoints and ie. vscode plugin could gray out or put horizontal grey bar for code that's shaken. It would be great plugin.

onion2k• 5 years ago

Something that drives me bonkers about current tree shaking is that there’s no good way to see what got shaken.

On a super basic level you can just build with and without tree-shaking and diff the results. If you turn off transpiling and minification, and move your bundled NPM modules to their own separate file, you can see what parts of your own code got shaken out pretty easily.

I'm tempted to have a go at writing a tool to do that actually. It would be useful.

jgalt212• 5 years ago

diffs (and network graphs) seem like a great idea, but they quickly get so big that one can't make heads or tails of the visualization.

barbazoo• 5 years ago

TIL about tree shaking and that that's a thing in JS. Is that applicable to other languages? Why not Python?

andrewstuart2• 5 years ago

It's common in javascript at least because you're delivering the code across the network, frequently, and at dev time, you're probably pulling in large libraries of components and frameworks that you may not use in their entirety. Since the load time is one of the core metrics used by SEO, and has shown to have a significant impact on conversion rates, it's worth the investment to remove code that's not being used.

I'm sure there are other places where it's applicable, but modern javascript and webapps have to be the quintessential use case.

avbor• 5 years ago

Most other compiled languages do dead code elimination, which sounds similar but is a little different. Think of dead code elimination as removing code that doesn't change the output, while tree shaking instead includes code that could run.

To apply this to python is interesting - if you were creating a packaged version, I could see "compiling" the code to a separate package with only the required imports.

IshKebab3• 5 years ago

Think of dead code elimination as removing code that doesn't change the output, while tree shaking instead includes code that could run.

Those are both dead code elimination. Webpack even says:

Tree shaking is a term commonly used in the JavaScript context for dead-code elimination.

littlecranky67• 5 years ago

I know that Microsoft is doing some kind of treeshaking in Blazor (basically a .NET runtime in WASM), since they do not want to ship a full .NET base library to the browser. Not JS related, but the problem is the same - you want to ship as little data as needed on websites that are dynamically loaded.

IshKebab3• 5 years ago

Yes it is, it's just that Javascript has a stupid name for it. In other languages it is called dead code elimination.

esperent2• 5 years ago

They are not the same, and the term comes from Lisp in the early 90s, not JavaScript. Tree-shaking is actually "live code inclusion" - in other words, it approaches the problem from the other direction.

https://en.m.wikipedia.org/wiki/Tree_shaking

raxxorrax2• 5 years ago

Is that applicable to other languages

It could be, but I don't see it as too practical as dependencies are very likely quite transparent and don't themselves rely on many dependencies. But for JS?

pwdisswordfish8• 5 years ago

s/for JS/on NPM/

There's no reason you can't write JS the same way you would write Java or C++. (And if you were writing "serious" applications in JS that weren't web apps before NodeJS came along, you did. Of course, you still can, too.)

geokon7• 5 years ago

Anyone have any experience using a Java treeshaker? I have a Clojure GUI/JFX App that's bloated and I've only managed to slim it down with manual package pruning.

I couldn't get Proguard to work with Clojure. Granted proguard is quite baroque so I probably did something wrong. But maybe there is some simpler solution out there for the JVM? Or would this simply be impossible b/c Clojure uses reflection? Though so does Javascript from what I understand..

setr• 5 years ago

Shouldn’t the JVM already be doing this, as part of dead code analysis during the compile step? I believe tree shaking in JavaScript also only handles static analysis — eg imports and function calls.

I imagine though usage of something like exec throws DCE right out the window, poisoning the whole program

geokon7• 5 years ago

No, the Java compiler does not do this. You can try it yourself and see. All dependencies are bundled into the JAR. If you have some large library as a dependency (like BoofCV or JavaFX for instance) then all of the classes and dependencies are bundled. Even if you don't use 99% of them. That's why a lot of Java libraries are broken up into many smaller sub-libraries which you can then manually pick and choose from. It's also why you sometimes see people re-implement smaller sections of larger libraries - b/c you don't want to be forced to drag in the whole mess of code

As far as I understand, and I'm murky on the details, but if you're language supports reflection then you can call any dependency during runtime. And so this is not amenable to static analysis. So the Java compiler doesn't have the same guarantees as say a C++ compiler.

Most people are running JVM on the server so they just don't care about executable size (and on Android people use proguard). But reflection in general is a source of headaches..

If you use something like GraalVM then it will try to prune dead code but it will not work well with reflections.

That all being said, on the specifics, I'm actually not entirely sure how you'd use reflection to arbitrarily call library code at run time. If anyone knows, I'd be curious to see an example

kaba03• 5 years ago

I recommend checking out the jdeps command line tool, that comes bundled with the JDK. Though you probably have to read up on modules a bit first. It can create very lean “JREs” that can be bundled with the program and only load the required classes.

geokon7• 5 years ago

As I understood it, jdeps is only for shrinking the runtime and bundling it with the program. This part seems fine.. or at least they're doing the best they can :)

I ended up skipping this step and distributing a JAR - which is a bit of no-no .. but i just ask user to install the Java runtime. (it's better than making executable for every platform and then spinning up VMs to test)

However, jdeps or not, the rest of the executable still remains as bloated as before

esperent2• 5 years ago

I suspect (although I haven't checked the code) that this is actually a "dead code elimination" algorithm which is easier to write than a tree shaking algorithm.

Tree-shaking is actually "live code inclusion", the opposite side of dead code elimination.

https://en.m.wikipedia.org/wiki/Tree_shaking

markl42• 5 years ago

Might be worth checking the code but

tree shaking eliminates unused functions from across the bundle by starting at the entry point and only including functions that may be executed

Is what this code (tries to!) do

shwestrick• 5 years ago

"opposite" is not the right term... tree-shaking appears to be a particular technique for dead code elimination.

I.e., there are many algorithms you could use to eliminate dead code, and tree-shaking seems to just be a simple one that works well for JS.

somehnacct37574• 5 years ago

Will the need for bundling and tree-shaking still be so great with HTTP/2 and js module support in browsers?

danielEM3• 5 years ago

3 x Yes:

1. Because of lower overall bandwidth (in many cases it is a SIGNIFICANT difference between code size before and after treeshaking)

2. Because of better startup performance - less code to process, less pings to the server for new modules, less load on servers too (streaming one file vs streaming hundreds of them separately)

3. Because of better performance (eg. removing conditional branches that will never execute)

;-)

somehnacct37574• 5 years ago

I don't think you understood the question. HTTP/2 allows for one connection to the server to stream multiple files. There's no more network setup penalty for making a request. So all that remains is downloading the files. Now it doesn't matter if they're in one bundle or 20 files.

You also don't have to tree shake up front because the execution will do it at runtime. If you deploy some unused modules, the browser will never request them and the server will never send them.

For the remaining benefits I can think of - more bundle bytes means more optimal compression; or shaking out functions within a module - I'm not so sure the benefits outweigh the complexity of maintaining and using bundlers. We could be splatting the src directory straight to the server and letting HTTP/2 do the rest.

Tade0• 5 years ago

I love the function and variable names - straight to the point, without trying hard to be too precise.

tonyedgecombe• 5 years ago

The interesting stuff seems to be going on in https://github.com/facebook/jscodeshift

tyingq• 5 years ago

How does treeshaking deal with error paths (or other uncommonly used code paths) that aren't often used, but are important not to throw out?

ygra2• 5 years ago

The same as it deals with common code paths: What can't be guaranteed to never be called, it left in. E.g. you can remove functions in a module that are never imported anywhere else and never called, rather easily. You can also remove branches that are unreachable, e.g. inside an if (false). But for things like

    function f(s) {
      if (s === '') throw new Error()
      ...
    }

there probably are very few tools that will statically be able to tell that f is never called with an empty string and thus the validation logic is not needed. It's JS after all and tools that mangle it all fall on a spectrum between doing very little, but very safely and trying to do too much and breaking code.

Tiny Treeshaker: JavaScript tree shaking in 200 lines of code

Related Stories