Maven is broken by design

http://blog.ltgt.net/maven-is-broken-by-design/

by richardw • 13 years ago

143 89 13 years ago

blog.ltgt.net

Incremental builds: I am working at a Java shop at the moment and was amazed to find that even the resident Maven experts clean before every build. To me incremental builds are one of the most important functions of a build system, but it seems that users of Maven and other well-known build systems used in the Java world have given up on it because it just doesn't work. So anything which fixes even only this problem would be a step in the right direction.

jpollock• 13 years ago

I read how TeamCity did their incremental builds and wrote my own script. I drop an anchor, detect changed files and then build up a list of modules that need to be rebuilt. I then tell maven to build those modules and their downstream dependencies.

Speeds things up a tonne.

http://blog.jetbrains.com/teamcity/2012/03/incremental-build...

koevet6• 13 years ago

I don't understand why this is still a matter of discussion after so many years of blood and tears. Maven is dead, everyone else is moving to Gradle (Spring, Android, Hibernate, etc.).

I have been using Gradle now for almost 3 years and I can't be happier. The support is great, they roll out new features at a regular pace, it's fast and incremental build WORKS.

If you work in Java and you are still stuck with Maven, please, take a look at Gradle.

auvrw2• 13 years ago

If you work in Java and you are still stuck with Maven, please, take a look at Gradle.

or ivy? i really haven't spent a lot of time toying with the various build systems. i tend to use ant until manual dependency management becomes ... unmanageable, then maven because i've apparently been brainwashed (i.e. i heard that they use maven at a lot of not-so-bad companies like thoughtworks).

after the article, i do like the suggestion of ivy+buck, neither of which i'd paid attention to before below the clamor of gradle trampling upon maven. before starting considerations along the lines "this software does not do X" or "it does X wrong," what about asking "what's a reasonable task for a single tool?" or more specifically,

"do we really want dependency management and builds handled by the same program?" my impulse is always to go for things that are as modular as possible, but perhaps the reflex response of "no" isn't correct for some reason obscured by lack of experience with the particular problem domain.

ajanuary• 13 years ago

Ironically the biggest pain with Ivy is that 99% of your dependencies will be pulled in from Maven oriented sources, so you still have to deal with Maven's deficiencies around dependency configurations.

koevet6• 13 years ago

Ivy is "only" a dependency manager, you can't build stuff with it. Gradle was originally based on Ivy but they decided to roll their own Ivy/Maven compatible dependency engine.

Gradle is a very powerful engine: we use it not only to build Java/Scala/Groovy code but also to drive our continuous integration/deployment cycle.

Again, it obviously depends on your requirements, but if you have to deal with large and complex projects, with lots of submodules to build, do consider Gradle.

clumsysmurf• 13 years ago

Actually, according to the May 2013 Thoughtworks Tech Radar at

http://www.thoughtworks.com/radar

In the Tools category, Gradle is "Adobt", while Maven is on "Hold". You can read the full report but I'll exert the relevant section:

"Language-based build tools like Gradle and Rake continue to offer finer-grained abstractions and more flexibility long term than XML and plug-in based tools like Ant and Maven. This allows them to grow gracefully as projects become more complex."

They had a more elaborate explanation in previous editions.

Randgalt4• 13 years ago

No, everyone is not moving to Gradle. Gradle is Ant in Groovy clothing. If you want incomprehensible build files ala Ant, choose Gradle. If you want standard build files and a rich ecosystem, choose Maven.

I hated Maven until I tried the alternatives.

koevet6• 13 years ago

incomprehensible build files ala Ant

Please, explain incomprehensible (or show an example)

twic2• 13 years ago

Yeah, no. Gradle is really nothing like Ant. A bad build script author can use it to write build scrips that look like Ant with curly brackets, but then you can write COBOL in any language. If you take the ten minutes to learn how Gradle actually works, then you don't end up with scripts that are like Ant.

Advocating Maven over Gradle is just a joke. I routinely do things in Gradle scripts that would require writing an whole new plugin to do in Maven.

vorg• 13 years ago

Gradle is Ant in Groovy clothing

Gradle's purpose is to muscle in on the consulting / conference market for Ant, just as Grails's purpose as wrapper around Spring and Hibernate was initially to muscle in on their consulting businesses. Now they've hooked in enough users, their bait and switch business strategy is now to screw them for as much money as they can.

So if you use Gradle now, you'll pay the cost later on after Gradleware is sold to some GE or VMWare or EDS or Oracle.

tmarthal• 13 years ago

Even for strict java builds, I've been using 'sbt' (the Scala Build Tool). It really is the successor to gradle, in many regards. Not sure you've tried it as one of your alternatives, but I'd implore you to give it a try.

dschiptsov• 13 years ago

Just maven or the whole Java philosophy of avoiding any understanding/researching/thinking before beginning to pile it up inside an IDE?)

cuillevel3• 13 years ago

I don't think there is an actual maven IDE. You normally use it from the command line, since it's integration into IDEs is serverly (eclipse...) lacking.

chii2• 13 years ago

If you use Intellij IDEA, their maven integration is top notch. light years ahead of eclipse.

smrtinsert3• 13 years ago

I don't use maven for my builds these days but that's completely wrong. configuring deps is painless as is adding plugins. I've used it to build maven projs as well as osgi projs.

liveoneggs• 13 years ago

netbeans can use maven out of the box

bun-neh• 13 years ago

I wouldn't call it the "Java" philosophy. Rather, I would call it the "wet behind the ears" philosophy. It just so happens that they correlate because of the way people are trained.

GlennS• 13 years ago

Running joke in my office: whenever you waste a day on some trivial task because you had to edit some stupid XML config file, you lose a sanity point.

I have learned the hard way to read some of the source code of any external library or tool, no matter how widely accepted, in case the person who wrote it didn't attend the class on not using global variables or depending on undocumented internals of the JVM (I'm looking at you Log4j 2). Or in case they were just plain bonkers, like everyone who was involved in writing Spring.

brown9-22• 13 years ago

To suggest that this is an effect of the language rather than various teams of different people building different tools at various points in times seems silly.

What other language hasn't had multiple build solutions that were each "the one solution" at some point in time, only to be supplanted later?

How many build solutions are there for C++, or Ruby?

chopin• 13 years ago

I am not very experienced in using Maven, but the biggest problem I have with it is, that it is hard to tell (at least for my level of experience) from where dependencies are pulled. Is it my local mirror, is it something on a central nexus in the company or is it totally from outside?

Build reproducibility, which the author mentions is another issue. When I scrub the local mirror, the outcome can be completely different. Previously builds where working fine, when I scrub, everything breaks. This is mainly due to non-local dependencies which had been cached locally, but not replaced by most recent versions.

Another suspicion I have is that most build scripts are slapped together by googling until it works. Which of course is prone to being fragile (not to mention legal issues because it is opaque from where all stuff is pulled).

fs111• 13 years ago

Never, ever, ever add any repositories to your pom.xml. Never. Have one central repo (nexus, artifactory, whatever) for your organization and make it a mirror of all others via settings.xml, not in the pom. This adds a lot to build stability and reproducibility.

richardw• 13 years ago

He and others continue discussion here, with some depth.

https://plus.google.com/113945685385052458154/posts/7LHoGXQd...

hansgru2• 13 years ago

Maven is broken by design

It might be, but it still has it's merits. It was one of the first, and many other improved tools can't work without it's repositories.

I still like the IDE support for MVN: e.g. IntelliJ draws a nice dependency diagram that I find useful, even if I don't use Maven for something else in a project.

Maybe it's biggest mistake was that dependencies are global by default, and not local (like Node's NPM). If the dependency structure would have been local to a project, than many of the pain points I encountered with MVN in the past wouldn't have even existed.

mercurial2• 13 years ago

Maybe it's biggest mistake was that dependencies are global by default, and not local (like Node's NPM). If the dependency structure would have been local to a project, than many of the pain points I encountered with MVN in the past wouldn't have even existed.

I don't understand the local vs global dependency thing. What does NPM do differently from Maven?

delluminatus• 13 years ago

NPM stores dependencies in a node_modules folder within the project directory, i.e. "locally." It does have a command-line flag to store something globally, though -- useful for dependencies with executables.

I'm not a Java developer but it sounds like Maven stores all its dependencies globally (like in /usr/share/java). I'm not sure why that would be a "pain point" except making it hard to do different versions of the same library.

16bytes• 13 years ago

Maven stores all of the artifacts that it retrieves in one big local cache so that you don't have to pull the same dependency down.

NPM, by default, will store the artifacts into a directory local to your project.

The maven way sounds nice, but it's easy to possible to screw things up between different projects. (Especially if you have mvn 2 and mvn 3 projects, ugh).

With NPM, everything is nice and separated.

Ivy is like a mix of the two (a global local cache, but pulls down to the local project), which IMHO, has the benefit of neither while getting the drawbacks of both.

hansgru2• 13 years ago

I don't understand the local vs global dependency thing. What does NPM do differently from Maven?

As others have pointed out, NPM as a default is using "local" dependencies, (something I couldn't achieve with Maven easily).

"Local" means that the dependencies are in a sub-directory of the project, and that the transient dependencies are in sub-sub-directories too if needed.

With NPM, for us some of the implications of this are the following:

- being "self contained" I can check in a VCS and tag it "completely" or just zip it and send somewhere else, where it can be built "as it is", even on CI machines or production machines that have no Internet connection, nor do require a company internal Maven server.

- bugs and issues with building are reproducible. The "it builds on my machine" doesn't happen anymore. (Just search the different Apache mailing lists of how often this happened even in those open source projects). We had this problem very very often - to the point where several projects were migrated to ANT again.

- the availability of public Maven Repos, their mirrors and their sync is just bad (sometimes it's the corporate firewall/proxy the problem), so builds just fail all to often. Many companies don't use internal Maven Repo Mirror

- none where I worked had one, nor could we convince the management to finance one.

- very often (even visible in open source projects that build with Maven), there are some dependencies (maybe not from the start of the project, or maybe only temporarily) that are not available in the official repos, so need to be manually added. Doing this in a sub-directory is much easier, and it needs to be done only once, so others will just use it.

These are just a few issues that NPM seems to have been solved nicely (or just not having them because of of the "local" defaults).

Of course, there are also disadvantages of this "local" approach (like the redundant disk space usage - where the global one is very efficient), but from my experience all these disadvantages pale compared to the problems and frustration Maven brought in many projects that adopted it.

chii2• 13 years ago

i might be mis-interpreting your local vs global dependencies, but if you are wary of the network nature of maven, you can always check into your scm the maven repository (and perhaps, pre-fill it with just the deps needed).

hansjorg• 13 years ago

a pom.xml can only list a single source folder. If you have more than one (for whichever reason)

I agree that this example demonstrates that the Maven model is broken, but why would you structure your project like this rather than using another module or a submodule?

Maybe it's just lack of experience, but I highly prefer the rigidity of Maven projects compared to some of the messes that more expressive build systems allow.

hrjet• 13 years ago

One thing which I detest about maven, it uses http by default. Getting it to use https requires a lot of work[1] so many don't use it.

By default, not only are you downloading a truck load of jars from the internet and running them locally, you are fetching them over an insecure channel!

[1] http://maven.apache.org/guides/mini/guide-repository-ssl.htm...

nl• 13 years ago

Honestly, so what?

What is HTTPS supposed to protect against in this case?

1) If you are using it to protect against snooping (why?!) then you must realise that the logs of many Maven repositories are effectively public (eg, I believe Apache committers can access the repo.maven logs pretty easily). For protection against snooping, use a local repo.

2) If you are expecting it to protect you against tampering in transport, then you are better off using a local repo (again). It's much more likely artifacts are tampered with at the remote repository rather than during the transport phase.

michaelt• 13 years ago

It's for when you trust the public repo operator, you trust the authors of the package you're downloading, you trust your local machine, you trust the repo operator's SSL CA, but you don't trust the connection from your computer to the public repo.

For example, if you're using dodgy public wifi.

andor• 13 years ago

It's much more likely artifacts are tampered with at the remote repository rather than during the transport phase.

MITMing the public wifi at some coffee store is much easier than breaking into the official Maven repositories. At least I hope so. That's why RPM and DPKG packages are signed.

rbanffy• 13 years ago

3) If someone poisons your DNS server and points you - and you alone - to a hostile version of the jar you are downloading.

dlitz• 13 years ago

https is useless for package management, given that most people are downloading from mirrors. What you want is a signature over the entire repository, like Debian and Ubuntu do (not just signing individual packages, like Fedora does). Once you have that, you don't need HTTPS.

tomohawk• 13 years ago

Switched to buildr (http://buildr.apache.org/) years ago. So much better than maven.

stephen2• 13 years ago

+1 for buildr. It admittedly takes awhile to learn the DSL, but it seems like all of these DSL build tools take awhile to learn.

(I think that was one really nice thing about Ant--you could scan their manual/list of tasks and understand it very quickly.)

It seems a little odd using Ruby to build Java projects, but ironically I actually prefer that over Groovy.

This is horrible for me to say, but I'd like Groovy to gracefully go away--it lost the Java.next race (they were stubborn about dynamic typing for way too long) and I feel the only reason it's still around are for the admittedly nice frameworks it sprouted: Grails and Gradle.

The language itself is "meh", IMO.

...although it bothers me to no end that a dynamic language like Groovy had compile-time AST translation way before other (similarly mainstream) static languages like Scala.

Anyway. Gradle's momentum is impressive. Perhaps I can unlearn my Groovy bias.

Communitivity• 13 years ago

I'm a sw engineer programming Java since 1996. I started using Maven when it first came out as the declarative build language hooked me, and became the local Maven expert. I've watched it grow with pride, and sometimes in anger. I still believe a declarative build language is best, but it won't be in XML. I suspect it will eventually be in something like OWL, whatever comes along after OWL that fixes its issues.

So as of a few years ago I too moved on to Gradle and have loved it.

But...

Language based build systems offer more expressiveness, but that is also more rope to hang yourself with if you get someone that doesn't know Gradle well doing tinkering with your build. For example we had someone add a custom checkstyle report to an Android project using Gradle. After their edits it stopped working because of the way the Android plugin was designed, bringing the build down for several hours. I came in, rewrote the Gradle build file to workaround the issue and it worked. Then I documented to the person why it hadn't worked and what I did to fix it.

As with the adoption of anything that gives you more rope to hang yourself with, it's a necessary piece of Gradle adoption to document your practices and train your team well.

twic2• 13 years ago

"I documented to the person" - that is a euphemism i will have to remember.

Yeah, I documented it to Jim. That's a documenting he won't forget in a hurry, make no mistake.

ssfsx17• 13 years ago

What is inherently virtuous about a build tool being pure-declarative and having no imperative impurities?

andrewaylett• 13 years ago

Any tool can get hold of the full state of how to build the project, then interpret it itself, rather than having to work out what the imperative code is trying to achieve.

ExpiredLink3• 13 years ago

Do it the Maven way and you'll have no problems.

Fight the Maven way and you'll end up in a mess.

tl;dr Understand the tools you use.

ghostdiver2• 13 years ago

Maven killed some great open source projects, i.e. Apache Cocoon

ExpiredLink3• 13 years ago

According to the hp Apache Cocoon is based on XML/XSLT and Spring. If it failed I bet not because of Maven ...

jebblue• 13 years ago

I've tried on many occasions to make Maven work with Eclipse but Maven nearly always finds a way to mess up Eclipse's excellent project management abilities. I'd say that if you just write code in an editor then Maven makes a lot of sense, if you use a powerful IDE like Eclipse then Maven becomes an unwieldy and unnecessary challenge.

ebiester2• 13 years ago

That's fine, so long as the problem you're trying to solve is easily expressed within Maven's structure.

And sometimes, expressing it withing Maven's structure can lead to very slow builds, at least within my experience.

But perhaps I'm prejudiced; I hate maven, and have since my first experience with the Mvn 2 betas. You know when you see a new technology and you say, "this is painful now, but it's going to be worth it in the end?" I got the "this is painful now, and is only going to get worse as plugins depend on crazy ass behavior" feeling.

From everything I've seen since, I was right.

eropple2• 13 years ago

From everything I've seen since, I was right.

Please substantiate this, as I've been using Maven for quite a while and while it isn't perfect, it's not remotely painful. And I'm not talking about just for "build this jar" projects; most of my Android apps use Maven, too, as do a number of projects of mine that have native dependencies.

(I have no problem with Gradle, either; I just generally find most of the Maven complaining to be overblown.)

timr2• 13 years ago

Reading Java people's complaints about the deficiencies in their fifty different build tools makes me want to force them all to use Make.

angersock• 13 years ago

They typically spell it "ant".

ExpiredLink3• 13 years ago

That's the main problem people have with Maven. They try to use it like make/Ant - and fail miserably.

timr2• 13 years ago

Yeah, they misspelled it.

Randgalt4• 13 years ago

FYI - http://tesla.io/tesla/index.html

Maven is broken by design

Related Stories