Cargo, Rust's Package Manager

http://crates.io

by wunki • 12 years ago

229 120 12 years ago

crates.io

wunki2• 12 years ago

And here is the announcement from Yehuda Katz:

https://mail.mozilla.org/pipermail/rust-dev/2014-June/010569...

yazaddaruvala• 12 years ago

Why not just use JSON?

Infact, why not just use npm's package.json?

Svip• 12 years ago

I may not be a majority here, but I see JSON as a data transportation format, while I see TOML or YAML as configuration formats.

You cannot write comments in JSON, for instance.

progx• 12 years ago

Example from the website in json

    {
      'package': {
         'name': 'hello-world',
         'version': '0.1.0',
         'authors': [ 'wycats@example.com' ]
      },
      'bin': {
        'name': 'hello-world',
        'comment': 'the name of the executable to generate'
      }
    }

So where is the problem ?

And you have the advantage, that other tools can use the complete file including the comment. In TOML you need an extra parser to grab the comment.

riquito• 12 years ago

You're not alone. YAML or similars are the right formats for these things, not JSON. Less clutter, comments separated from data, easier to read (by humans).

bryanlarsen4• 12 years ago

YAML is a serialization format, not a configuration format.

pandatigox• 12 years ago

I think that's a good point. Why use .toml when we have better alternatives already?

(I admit I have a bias against toml, but still...)

the_mitsuhiko• 12 years ago

Which ones would that be?

cmrx642• 12 years ago

Because comments.

cmrx642• 12 years ago

There isn't a lot of attachment to the exact manifest format used, but using json is clearly a step backwards!

baq• 12 years ago

json is only marginally easier to edit by hand than xml. also, comments.

jdeseno• 12 years ago

The read-manifest command seems to output a JSON of your Cargo.toml if you have tools you'd like to use that require it:

    cargo read-manifest --manifest-path .

illumen• 12 years ago

Yeah, why use some format that no one knows, and is harder to parse by other tools?

nirvdrum5• 12 years ago

Not only that, but TOML is still considered unstable by its author. Granted, it hasn't been updated in several months. But that seems like a shaky foundation to be building on top of.

From: https://github.com/toml-lang/toml

"Latest tagged version: v0.2.0.

Be warned, this spec is still changing a lot. Until it's marked as 1.0, you should assume that it is unstable and act accordingly."

zimbatm• 12 years ago

JSON is annoying with the commas. One forgotten comma or comma too much and the whole file doesn't parse.

tomlu• 12 years ago

How do you pin a dependency to a particular version or git sha? I can't find anything in the FAQ or docs that implies that it's possible.

cgag• 12 years ago

From announcement email:

The next features we're planning on working on are: - Supporting refs other than `master` from git packages

sp332• 12 years ago

As a workaround, I guess you could fork a specific version on your own github account, and use that as the dependency?

kibwen3• 12 years ago

The Rust folk are adamant about supporting versioned libraries out of the box, so support for this will definitely land before 1.0.

jpgvm• 12 years ago

I would really love to see some docs on how to actually install and get started with Cargo.

It doesn't ship with Rust and the docs on GitHub and crates.io are not very enlightening.

wunki2• 12 years ago

This was how I installed it (Mac):

  1) Install latest version of Rust found here: http://www.rust-lang.org
  2) git clone --recursive git@github.com:rust-lang/cargo.git
  3) make
  4) make install (could be that sudo is needed for you)

buster• 12 years ago

I'm putting rust in $HOME/bin (and other self compiled software). You can change the default installation from /usr/local by running

  DESTDIR=$HOME make install

btd• 12 years ago

There is no brew formula?

steveklabnik19• 12 years ago

Yesterday started my contract with Mozilla to write documentation. Today, I'm starting on re-writing the tutorial: https://github.com/rust-lang/rust/pull/15131 (apparently, bors is a bit backed up)

The new tutorial will be based around 'real' Rust development, and so will assume Cargo.

That said, http://crates.io/ should have install instructions on the site. I'll open a ticket and get on that.

ithkuil2• 12 years ago

I'd love if

$ cargo cult

would build a new project/module from a template.

tempodox2• 12 years ago

Yes, absolutely. Cargo cult doesn't have the best reputation, but this would be an opportunity to cherish it :-)

steveklabnik19• 12 years ago

The plan is for that to be `cargo project`, but that might be a fun alias. :)

FrozenCow• 12 years ago

I haven't been following the development of this package manager, but previous attempts at making a package manager for Rust have failed. Is this package manager supported officially now? I really hope it will stick around.

dbaupp• 12 years ago

Yes, the developers are actually domain experts being paid by Mozilla. The release of this website also coincided with the move of the source repository into the rust-lang organisation: https://github.com/rust-lang/cargo .

bjz_4• 12 years ago

Yes, it will be supported. The Bundler guys were specifically contracted by Mozilla to sort out the mess. They are actually using Rust in production at the moment, so the have a big stake in its future.

The quiet nature of the development process of Cargo was actually a response to the previous package management failures. The idea was not to publicise heavily before it was ready for dog fooding. This seems to have paid off.

mellisarob• 12 years ago

This is good news. a better alternate to conventional techniques

tempodox2• 12 years ago

This is very welcome news, indeed. I will have to give it a try as soon as I can make the time.

I hope it will be more stable and work better than the Haskell package manager, Cabal. I literally never got that to work on any machine. It would typically destroy itself while attempting to update itself...

Sanddancer4• 12 years ago

Will this play nicely with the package management tools OSes already have, or is this going to end up being yet another source for files/packages to accumulate that are outside the view of well-documented and designed administrative tools?

sanderjd5• 12 years ago

I think it would be just as easy to package rust programs using OS native package management as anything else, and I'm sure packages will be made for popular things for common package managers. But OS native package managers are a royal pain to use when actively developing on a project that has some dependencies, and bespoke Makefiles are a very imperfect solution.

Flipping your question around a bit: will package manager creators and maintainers ever develop better solutions to the use case of development rather than system administration so that we don't have to keep creating these language-specific tools?

kibwen3• 12 years ago

This is an insightful point. Nix is the only package manager I'm aware of that seems like it could fit the bill: https://nixos.org/nix/

Sanddancer4• 12 years ago

Have you taken a look any at the FreeBSD ports system? It uses makefiles to manage pretty much any software install you can think of. Additionally, there's a pretty good infrastructure there for managing custom build trees, etc. Additionally, they also have things like BSDpan that let you use arbitrary Perl modules with the package management system.

ithkuil2• 12 years ago

I had the impression this fills the same role as bower, mpm (non global) dependency or plain custom vendor dirs handled with your favourite git module paraphernalia: your build is the place your dependencies live (+ possible cache directory somewhere else). And, these you use this packaging system to manage your build, not the deployment of your (iirc statically linked) build artifacts.

While technically the cache directory is a place where files can accumulate outside the view of a well documented and designed administrative tools, this is common problem shared with many tools including your favourite browser.

alextgordon• 12 years ago

Ha! There is nothing well-designed about the mutually incompatible, political hell that is OS package managers.

The best solution (as taken by npm, virtualenv and others) is to install libraries locally to the project that is building them.

That way, package management becomes the sole concern of the build system.

"Accumulation" is a good thing, it means each project has the exact version of a package that it was tested with, not some later version that a sysadmin decides it "probably should work with".

bryanlarsen4• 12 years ago

OS package managers make two assumptions:

- that packages follow semver

- that the OS packagers are in a better position to test package combinations.

If the author releases a new version of libfoo, and A, B and C in an OS repo depend on libfoo, then the OS packagers do not release a new version of libfoo until the tests for A, B & C pass.

These are two good assumptions, and the language package world would be in much better shape if they followed those assumptions too.

Sanddancer4• 12 years ago

Which is why we have testing and staging environments to make these changes on first. Conversely, when everything is pulling in their own version of a library, you run into situations like the zlib buffer overflow, where you had huge numbers of programs that needed to be rebuilt because no one used system-packaged libraries. Obsolete and vulnerable software is a liability, and not having tools which can readily tell what software needs to be updated makes quite a few peoples' jobs quite a bit harder.

awj3• 12 years ago

Until the various OS package managers stop pretending they are a universe unto themselves, this problem will continue. Languages have to distribute libraries across all of their host systems, so focusing on support for any particular package manager (or small subset thereof) doesn't buy much.

Hopefully the cargo team can come up with a solution that works a little better here, but I wouldn't hold my breath.

dmacvicar• 12 years ago

Each language having its own tools is fine and I doubt OSes are against that. Those tools just need to support a very specific set of features or scenarios to make packaging easy:

Understand that enterprise OSes are not built on developer Macbooks. Enterprise distros have reproducible builds on automated systems with chroots that contain only what the package needs, no network access and sometimes the build happens inside a virtual machine. Its is almost ridiculous how after maven forgot the topic of being "offline" almost every tool released afterward has done the same mistake.

Understand that Linux distributions sell support and that means being able to rebuild a patched version of a failed package. So whatever dependencies are pinned in Cargo.toml or Gemfile is irrelevant. The OS maker will override them as a result of testing, patching or simply to share one dependency across multiple packages. Distros can't afford having one package per git revision used on a popular distro and then afford to fix the same security issue in all of them.

So having "cargo build" be able to override dependencies and instead look on the installed libraries with a command line switch or env variable helps the packager not having to rewrite the Cargo.toml in the package recipe.

Maven was probably the first tool that made packaging almost impossible and completely ignored the use-case of patching a component of the chain and be able to rebuild it completely.

Semantic versioning is great news, because it allows you to replace the dependencies for a common semantic version (not exactly the same).

For integrating with the C libraries not much needs to be done. If you support pkg-config then you cover 85% of the cases.

sigzero2• 12 years ago

If someone can log into github and enter a ticket to say no to toml. Yaml would be perfect for it and it is mature and people already know it.

steveklabnik19• 12 years ago

I don't have strong feelings about toml, but the YAML spec is incredibly complicated, and has way too many features for a config file format. And security vulnerabilities O_O

thinkpad20• 12 years ago

I don't know about the security vulnerabilities, but it works fine as a config file format (we use it at my company for a lot of in-house stuff). I had a similar reaction to the language. Even if not YAML, why not just use JSON? It's universal, dead simple to use and understand, has extensive libraries in just about any language, etc...

That said it's not that big of a deal. At least it's not an in-house markup like Haskell's cabal...

bryanlarsen4• 12 years ago

This is sweet:

For example, if I have three packages:

   - uno depends on json 1.3.6
   - dos depends on json 1.4.12
   - tres depends on json 2.1.0

Cargo will use json 1.4.12 for uno and dos, and json 2.1.0 for tres.

Hopefully rust builds a culture that respects semantic versioning better than the Ruby & Node cultures do. That has to start at the top. There were several Rails 2.3.X releases with minor ABI incompatibilities. Truly respecting semver would have required these patch level updates to get a new major number.

steveklabnik19• 12 years ago

Note that this is a sliiiightly modified SemVer: in SemVer, these three things would conflict. It would be strict SemVer if uno depended on ~>1.3, and dos on ~>1.4.

We hypothesize that this difference works better for AOT compiled languages. And since it's pre-1.0, it's all good. This is the 'we'll watch this closely and adjust' from the email: it might not be okay.

ionforce• 12 years ago

Why would uno not work on 1.4?

mercurial6• 12 years ago

What would be fantastic is if a Cargo repo could refuse a package which doesn't follow follow this policy (try to upload a new minor version with an incompatible ABI, get 400 Bad Request as a response). Of course, it won't save you from a change in semantics, but it would already be a good step forward.

kibwen3• 12 years ago

I'd be very curious to know if it's possible for an automated tool to perfectly detect ABI breakage.

Argorak2• 12 years ago

Don't equate Rails with Ruby (some projects follow semver very closely).

Rails follows a shifted semver version, as documented here: https://github.com/rails/rails/blob/master/guides/source/mai...

Bumps from 4.x to 4.y might contain smaller breaking changes.

For a large project as Rails, I find that reasonable, otherwise, we'd be at Rails 20 by now, which also doesn't quite give a good sense of how the project evolved.

bryanlarsen4• 12 years ago

Rails, for better or worse, is the flagship project of Ruby, and shapes the Ruby culture quite strongly.

I would much prefer Rails 20 than the current situation. If you want to make a major marketing level move, introduce a codename or something. Separate marketing from semantics.

awj3• 12 years ago

MRI didn't follow semver up until recently, and even now it has some caveats about it.

Ruby as an ecosystem doesn't really care for semver. Some projects follow it anyways, which I can respect, but they aren't the norm.

ripter• 12 years ago

Npm require modules to follow semver. https://www.npmjs.org/doc/package.json.html#version

Now sure why you think the culture there doesn't respect semver.

TheHydroImpulse• 12 years ago

Requiring a version number to be within the semver format is one thing; but abiding by the rules of semver is another thing entirely. You'll see plenty of Node packages break compatibility on minor or patch updates. It's not something I see as extremely important in that community.

oakaz• 12 years ago

I don't expect that since the version field is optional.

bfrog• 12 years ago

This looks like yet more awesome stuff coming out of the Rust camp.

I'm pretty excited to see Teepee and Rust come together so I can really give it a spin doing what I'm currently doing daily for a job.

pjmlp2• 12 years ago

What about binary only dependencies?

steveklabnik19• 12 years ago

This should absolutely be handled in some good way. I think right now you'd just set the script attribute to something empty, but that's obviously a hack.

Locke1689• 12 years ago

I think mixed is also very important. For example, sometimes you really do only have a binary dependency (e.g., .NET assemblies), while other times, you may have a binary + source dependency (e.g., library + header files).

cies• 12 years ago

Learn two things:

1. Wycats (Yehuda Katz) is on Rust apparently :)

2. `.toml` -- some crossbreed YAML/INI file format that I like

stormbrew2• 12 years ago

It'd be really nice if it didn't impose a hard choice of json-ish to use for the manifest, even though that's a rather small thing. Just don't want to be stuck with toml if it doesn't gain wider acceptance than it has so far.

dreamdu5t• 12 years ago

While I support semantic versioning, people need to be aware that it's only as good as the package maintainer. I have used packages that have (unintentionally) broke semver conformity. Nothing really stops an author from releasing breaking changes when going from "1.2.1" to "1.2.2".

wycats• 12 years ago

The goal is to use package manager defaults and community pressure to keep things on the straight and narrow. We'll see how it goes!

Cargo, Rust's Package Manager

Related Stories