Just-in-time packager for GitHub repositories

https://www.jitpack.io

by jitpack • 11 years ago

55 31 11 years ago

iadapter• 11 years ago

It's an interesting concept. Instead of you uploading binaries the server builds and serves them. Any plans to support other build systems?

jitpack5• 11 years ago

There are! Probably Sbt first and then Leiningen.

tekacs• 11 years ago

This is brilliant! Another vote for SBT!

jermo2• 11 years ago

Nice! This should make it a lot easier for developers to publish their projects.

maaaats• 11 years ago

I thought the same. Publishing small JS libs is easy, just drop a Bower file and register it. For maven I need to build it, sign it, get access to a central repo (can take days) and then finally upload it.

Of course, this means that the quality is often much higher, but I think the barrier is a bit too high as it is right now.

pron8• 11 years ago

For maven I need to build it, sign it, get access to a central repo (can take days) and then finally upload it.

Those are the steps required to host your artifacts on Maven Central. Hosting on other public Maven repositories requires less work, and if you host your own, you set the bar (you can even turn a GitHub repo into a Maven repo, and upload with a git push).

Having said that, if you've built something worth using, the extra steps required to publish to Maven Central are both negligible with respect to your overall effort, and are more than worth it (getting the initial access usually takes hours, or one day at most, and only has to be done once for the entire organization/author).

So far, the only projects I've seen that are not already hosted on Maven Central or any other public Maven repository (jcenter, Clojars, java.net and others), are usually short programming exercises or works-in-progress.

drewolson• 11 years ago

Does jitpack currently support repositories in Github Enterprise?

jitpack5• 11 years ago

Currently not. Actually you are the first to ask and it sounds like a great feature:) most likely that would mean you host jitpack on premises. It's something we don't provide yet but it is on the road map.

aroden• 11 years ago

Wonder how high the usage would be. If you have github enterprise setting up Nexus and having build process publish to it seems easier than this build server.

arturnt• 11 years ago

Love this project, almost surprising that nothing like this exists today for the maven world.

pron8• 11 years ago

It does. There are gradle/Maven tasks to upload your artifacts to public Maven repos, and they're easy to set up so that Travis publishes them whenever you create a tag.

ithkuil• 11 years ago

But this allows anybody to depend on something that somebody else wrote even if he forgot to publish (for whatever reason). It reuses the upstream github namespace instead of creating many artifact group ids when random people publish somebody's else package

encloser2• 11 years ago

Using source repositories as packages sounds like an awful idea. Not only do you have to deal with runtime dependencies, but also compile, test, and other dependencies. What happens if the tags are not numbers? What if a tag you are using is deleted?

Go ask the Erlang community about the issues they have with using GitHub repos as "packages". Here is a discussion about packaging: http://mostlyerlang.com/2015/01/27/054-packages/

bjfish• 11 years ago

I think you might be misunderstanding how this service works.

The Github repositories don't serve the packages.

Jitpack checkouts the repository code, builds it, and serves it like a normal maven repository. Here is a sample repository https://github.com/jitpack/maven-simple and the maven repository it is served from: https://jitpack.io/com/github/jitpack/maven-simple/0.1/maven...

jitpack5• 11 years ago

That's correct

encloser2• 11 years ago

I got that it is building a binary and basically placing it in a local repo. I also get that it is looking for a binary from a GitHub Release first, which is mildly better.

I still think solutions like this are putting your build process at great risk. You will have build issues that are no fault of your own and completely out of your control. Unless you clone the github repository and then use that as your source. And cloning will have issues and risks of its own.

EDIT: See pron's reply for describing the risks: https://news.ycombinator.com/item?id=9029870

jitpack5• 11 years ago

Thanks for the feedback and its good to hear your concerns. Yes, you can use strings as tags but we recommend using Semantic Versioning just as GitHub recommends for Releases [1]. At some point we may choose to only build tags that match semantic versioning, to be decided. Unlike using just GitHub as source repositories we have some over what we serve and that's the difference.

If you make a conscious decision to release, add release notes and a tag why would you delete it? Especially if you want others to use your project. Its just not part of usual release workflow. Right now you can still get the binary for a tag that has been deleted because we cache it. We will probably add restrictions to prevent building another binary with the same tag.

[1] https://github.com/blog/1547-release-your-software

NeutronBoy• 11 years ago

See @vathpela

https://twitter.com/vathpela/status/563808697151803392 "Apparently Intel's Galileo build scripts that they distribute to users just clone my damn grub-0.97 repo off github."

https://twitter.com/vathpela/status/563808755213557761 "Which isn't a thing I knew when I deleted it Tuesday." "

norswap• 11 years ago

Amazing, this was sorely needed!

pron8• 11 years ago

A very nice project that reduces the barrier-to-entry of Maven artifact publishing, and saves an open-source project author a couple of hours of annoying work, but I have two concerns:

1. It might cause clashes for repositories that are hosted on a public Maven repository. Maven artifacts have one name, and one name only. Now, they may have two (with versions that are not necessarily aligned, to boot). This can wreak havoc on transitive dependencies.

2. Because git tags can be deleted and re-created, this repository breaks one of the strongest guarantees made by all (public) Maven repos, namely, that once a release artifact is published, it can never, ever change. This makes artifact versions unreliable.

In short, release artifacts must be unique and must be immutable; this project breaks both. That will cause actual runtime bugs that are virtually impossible to catch (and are bound to cost much more than the one-time setup -- per organization! -- of Maven hosting).

The sort of capabilities and guarantees required for source code management (and are provided by SCMs like git) are not the same as those required of binaries.

teacup504• 11 years ago

3) This creates an insane MITM opportunity.

Not only are they spitting back opaque binaries, but they're doing so by running arbitrary and untrusted user code.

There are already single-command tools for releasing a project to Maven, including tagging the release, bumping the version number in the build file, building and signing the jars, and uploading the results to a Maven repository.

Given that, why would you SaaS trusted builds!?!

pron8• 11 years ago

Well, to be fair, any public Maven repo is a MITM opportunity.

jitpack5• 11 years ago

Thank you for the compliment, Ron. Glad you see an upside to JitPack:) Regarding your concerns:

1. If you add JitPack as a respository then Maven will still check central first. If the artifact is in central it won't bother going to jitpack.io. Also JitPack enforces your group name to point to your GitHub repo to avoid name clashes.

2. Yes, git tags can be deleted but you really shouldn't do that on a public repo anyway. And most of the time people releasing with tags don't change them.

JitPack is built for the common case and not for the edge cases. And we believe that just because there are rare ways to break the system doesn't mean we should keep a high barrier-to-entry. Sharing your work should be easy and it should be easy for others to try your work.

pron8• 11 years ago

Glad you see an upside to JitPack

Of course! It's very cool! But I don't think it can be used in production (or even in development of important projects).

1. Say I am the consumer and I use a library author's GH repo with JitPack. Then, when the author releases her library to Central/jcenter/Clojars/whatever, she's free to choose any group name she wants (JitPack can't force her to do anything), and once she does that -- and another of my dependencies transitively depends on that artifact -- I am very likely to get a bug that's very hard to track down. Alternatively, once she publishes to central under a different group name, you might choose to simply break my build, which is also undesirable.

2. Maybe people shouldn't, but they do it all the time. I've done it on occasion, and I'm pretty sure I'll do it again. When that happens -- there's another impossible-to-find bug.

JitPack is built for the common case and not for the edge cases.

Which means it's a binary repository that is undependable. And remember, those edge cases are out of the library consumer's control.

just because there are rare ways to break the system

Unfortunately, they are not rare. First, most (actually, virtually all) production-quality Java/JVM libraries are already hosted on some public Maven repo, and any library that becomes production quality is likely to publish the artifacts, so breakage 1 is almost certain. As to 2, you should really get the data from GitHub. I think you'll find that deleting and re-creating tags is a lot more common than you think.

But even if those cases were rare, they can be very, very costly. We're not talking about a breaking build, but actual runtime bugs that might take days to track down.

doesn't mean we should keep a high barrier-to-entry

True, but the barrier-to-entry for the author is already quite low; jcenter makes it almost nonexistent.

What you've done is something else: you've let the library consumer control the artifact, so you've lowered the barrier for them, which is great! However, by doing so you've ensured that the author will almost certainly inadvertently introduce really, really tricky bugs into the consumer's project, bugs that they can neither control nor track down.

Still, JitPack is a great way to try someones work-in-progress code, to see if it might be worth it to help them make it production-quality. I am sure to give a try some time.

Just-in-time packager for GitHub repositories

Related Stories