Faster Maven Builds

https://blog.frankel.ch/faster-maven-builds/1/

by based2 • 5 years ago

74 34 5 years ago

blog.frankel.ch

mey• 5 years ago

Laughs in horrible Gradle Kotlin DSL build times.

lmm3• 5 years ago

Yeah, don't do that.

pjmlp4• 5 years ago

If it wasn't for Google's sponsorship it would have been gone by now.

Android is probably the only platform whose conferences to this day have a sessions about improving build times, thanks Gradle.

When doing Windows CE, Pocket PC, Windows Phone 7, WinRT, Symbian, iOS, I never needed a gaming rig as development workstation.

kaba04• 5 years ago

It is android’s fault, not gradle’s.

kaba04• 5 years ago

Gradle is one of the fastest build systems out there (certainly the fastest for Java projects because the tight integration with the compiler), given people know what they are doing. Most often than not they will write build logic as configuration which runs each time.

But when configured properly, it actually has parallel builds, with fine-grained build dependencies, so only the actually changed classes will be recompiled.

vbezhenar2• 5 years ago

Java build systems are too archaic and complex at this moment. They should be purged and replaced with new simple tiny tool, compiled with GraalVM. Java could be compiled extremely fast, javac does not do any fancy optimizations, it just spews bytecode directly derived from AST. 99% of maven/gradle run is mindless abstraction overhead.

raducu• 5 years ago

Are there other build tools for other languages that are much faster for the same codebase/complexity?

vbezhenar2• 5 years ago

Cargo. Or just plain make.

bpodgursky• 5 years ago

You are coming from a good place but... please god no, not another build system.

They are complicated because they do a fuckton of things. Please don't give me another proof-of-concept build system which then turns out to do 5% of what I needa and have no plugins available.

Gradle is fine, just use it. It's not the best. It's OK. If you hate it, use Maven. It's also OK. Just copy-paste some answers off stackoverflow and move on... we'll all be better off.

iamcreasy• 5 years ago

I sometimes contribute to a jMonkeyEngine, it is an open source game engine written in Java. This was my workflow in the past: fork > clone > added engine feature > deploy locally > added local deployment to my game's Gradle build file to test the newly added feature.

I recently found out that I can just use Github cli to clone from upstream, and add one line in my game's Gradle build file to tell Gradle to use the cloned folder instead of looking at local cache. In Gradle world it is known as Composite build and, it has really make me like Gradle despite their convoluted build files.

chii2• 5 years ago

Maven is great as long as you buy into their philosophy of how builds ought to work - phases and actions happening at a particular phase.

If your build has really custom stuff, and you cannot fit your custom work into those phases (e.g., it interferes with some other custom stuff), then you have a maven problem that's hard to solve. One usually resolve this by writing a maven plugin, and that kinda sucks. But hey, at least if you do it, you can contribute this to the whole plugin ecosystem and thus, improve it for somebody else.

tehbeard• 5 years ago

Don't. Just don't.

It's already a clusterfuck in JS land with grunt/gulp/webpack/TSC/rollup/parcel/snowpack/esbuild/rome etc, do not bring that madness to java as well.

chii2• 5 years ago

that java madness has already happened...10 yrs ago!

maven is at the level of maturity where you don't need to even refer to it by it's version - it's just maven now (there's been two previous incarnations of maven, called maven 1 and maven 2). Then there's ANT, and there's a bunch of bespoke tools as well that i'm not too familiar with - ANT is still useful, but i would say that it plugins into the maven toolchain acceptably OK.

bitcharmer• 5 years ago

Could you provide examples of better solutions for other platforms? I've been doing software dev professionally in 5+ languages for the past 20 years and speaking from experience java has one of the best build/dep management systems to choose from.

matsemann• 5 years ago

Java build systems are too archaic and complex at this moment

I feel the opposite. For 99% of use cases, a plain maven pom file works. If you need something custom, you should think hard about why the existing workflow doesn't fit you before moving on. I think maven mostly gets hate from hip developers because it uses xml, but the tool itself works fine.

I've spent ages more getting simple js apps to build correctly. It "just works" in java, and has for ages (thus archaic).

jillesvangurp• 5 years ago

Parallelizing builds and tests is definitely useful. Not only does it make things run faster but it can force concurrency issues to the surface that you would otherwise not find until after you deploy the code. This works especially well with integration tests that each might take a few seconds to run. Most of that is IO so you can actually utilize more threads than CPU cores here and still expect a speedup.

However, it requires designing your tests such you can do this. One key trick is using randomized data and test fixtures. Another one is avoiding the overhead of initializing databases, etc. over and over again. Of course, all that stuff is useful in just about any language. But some languages make this easier than others.

chenxiaolong• 5 years ago

100% agreed re. bringing concurrency issues to the surface.

Another nasty one I encountered a couple years ago: After switching the build servers to using ramdisks, the app started breaking in subtle ways. Turns out there were two conflicting dependencies, but building on a server using ext4 and with the specific number of concurrent threads caused the "right" one to appear first the .jar's central directory.

Gotta add filesystem directory entry ordering to the list too.

the_alchemist• 5 years ago

Nice write-up, and good to see that maven also evolves. The use of the daemon is really significant for the build time. Is there any clue what would take 90 seconds everytime you execute the maven process? Is it the creating of the build graph and its dependencies? What would happen when I change my dependencies or module structure, does the daemon still stay on 30 seconds?

willvarfar• 5 years ago

For my projects, building the jar-with-dependencies takes forever. And as there is a zip file from the previous build just sitting there, it’s a real shame the system isn’t clever enough to examine and reuse vast swathes of it.

lmm3• 5 years ago

A jar-with-dependencies is not really working with the grain of maven - you're meant to use the "exploded" version and use maven to run with the right classpath. Caching and reusing build outputs sounds clever, but my experience with Gradle is that the cache wastes far more time than it ever saves, either by taking longer to search the cache than to rebuild the thing, or by causing nondeterministic misbuilds that take days to debug.

emmelaich• 5 years ago

Can't you just 'install' it then make it a dependency itself?

shrx• 5 years ago

It would be great if the mvnd authors provided some instructions about how to set up mvnd to be used in an IDE like Eclipse or IntelliJ on the github repo [0]

[0] https://github.com/mvndaemon/mvnd

dikei• 5 years ago

Imho, mvnd is a step in the right direction.

Next is how to make maven cache work more reliably and aggressively, as maven is still too slow at incremental build: A zero-change build can take seconds, while Bazel finishes almost instantly.

brown9-2• 5 years ago

I think it’s more of the fault of needing to invoke a number of plugins bound to different cycles, rather than the compiler deciding if it needs to recompile files or not. I don’t believe Maven’s domain model has any concept of incrementalism or caching - only the compiler does.

skinkestek• 5 years ago

Step 1: if you use Windows, just use anything else.

Even a Linux VM running on top of the same Windows will probably be faster.

(I have not tested exactly that but I have tested in dual boot configuration and I have tested .Net cli apps and seen them running faster under a Linux VM on top of Windows than it did natively on Windows.)

tomohawk• 5 years ago

We switched to buildr. Build scripts massively simplified and build times drastically reduced.

Faster Maven Builds

Related Stories