Show HN: Asdf Clone Written in Rust

https://news.ycombinator.com/item?id=34583080

by jdxcode • 3 years ago

79 35 3 years ago

news.ycombinator.com

I think that asdf (https://asdf-vm.com) was a great idea for a project. It helps consolidate installing and running different programming languages into a similar UX. It also is built with a plugin interface that makes it easy to build support for new languages.

However it is so slow. I was just testing `node -v` and it was taking ~900ms. That kind of overhead is completely unusable. My shell prompt uses runtimes inside of it for various things so this effectively makes every command take multiple seconds to complete.

So I rebuilt it in Rust but using the same plugin ecosystem so it should be a drop-in replacement. I also added a couple of features that I wanted from asdf (aliases and fuzzy-matching).

Let me know what you think! Just know that people have only been using this for a few days so if you see any bugs, they're likely not big hairy issues, just overlooked edge-cases and will be fixed soon.

https://github.com/jdxcode/rtx

danfritz• 3 years ago

Will definitely check this out! I'm a long time asdf user but I always struggle with the shims and long timeouts.

I occasionally hit 2sec until my node version is resolved.

digianarchist• 3 years ago

Glad to see you fixed what I think is a bad UX decision by asdf, using the subcommand `add` for plugins and `install` for runtime versions.

jdxcode7• 3 years ago

They nailed the back-end stuff. I would not have come up with such a good, simple plugin design. But yeah, there are several UX issues like this. Part of it is because it’s written in bash, but ones like this I really couldn’t explain.

I’m trying to fix them while also supporting their syntax where I can, muscle memory is strong. So I support “plugins install” and “plugins add” If you notice any aliases that you think should exist, please mention them.

mystickphoenix• 3 years ago

This looks great, will be working with this in the next few days. I've always had a bad relationship with asdf as it seems to break existing tool management in creative ways that I'm unable to untangle without uninstalling it completely (long story, much thrashing of keyboard and mouse).

moomoo11• 3 years ago

The alternative to asdf (at least for me) is to just quit programming. It does what it is supposed to do well enough because in years I have not found anything better or necessary to replace it with.

e12e• 3 years ago

... asdf as it seems to break existing tool management in creative ways

Do note that asdf re-uses existing tooling, so trying to combine ruby-build and node version manager and asdf with node and ruby plug-ins is likely to crash and burn (because you end up forcing existing tooling to fight with itself).

Not sure if this was what caused your problems - but my asdf-life got a lot more pleasant when just asdf all the things (that I use).

Does mean that things like vs code needs the right environment / local tool versions and language servers to work with your projects and vs code plug-ins.

DavyJone• 3 years ago

Nice! How long does `node -v` take now?

EDIT: OK, I see the main trick is to use PATH instead of shims.

rtx does not use shims and instead updates PATH so that it doesn't have any overhead when simply calling binaries

There is a good reason `asdf-vm` uses shims, and is that it does not have to interplay or worry about other tools that set PATH and tools that need a reference to an executable could be simply set to `~/.asdf/shims`.

Ill take it for a spin, but this choice might have a lot of consequences that are not easy to foresee. A good example is `direnv` which as you mention in the README now requires to be set in `.envrc` and then disable global `rtx` hook I guess.

bin_bash4• 3 years ago

direnv does cause problems, but there is a workaround that functions (calling rtx within direnv). The main problem is it's annoying since you need .envrc and .tool-versions file littered through your apps (or just .envrc with "export RTX_NODEJS_VERSION=18")

There are some short term issues but I think I can resolve them in a way that direnv can't since I know exactly what PATH variables I added and didn't add. (Just remove everything with ~/.local/share/rtx prefixes.)

I could also just implement shims like asdf. The performance cost would be negligible (2-3ms of overhead for me running `rtx exec -- node -v`). I hate shims because they break `which node` though.

trws• 3 years ago

The which issue is fair, but you could also consider either adding them to the shell as functions, for the wrapper effect but getting a path listing on where as part of it. Or making the rtx binary respond to argv[0] being different names such that the “wrappers” are just symlinks and save the extra exec. None of these is perfect, but they avoid the issue where path becomes too long to manage, and the function option can make them completely independent of PATH.

Fire-Dragon-DoL• 3 years ago

The reality is that shims also cause many problems... So, i guess, choose your poison?

letmeinhere• 3 years ago

I like the idea, maybe I'll try it soon. I've recently been using asdf-direnv, configured to switch environments once, manually, and so taking shims out of the $PATH. Definitely giving up a bit of the magic for speed, so not sure if I love the tradeoff.

bin_bash4• 3 years ago

my issue with that solution is it makes directory changes with `use asdf` in the envrc slow. Of course that's not as bad as making all of the runtimes slow, but still intolerable IMO.

nicoburns• 3 years ago

Ohh, this interesting. I love most things about asdf, but not the fact that it's great big heap of bash scripts. I wonder how hard it would be to get this working on (non-WSL) windows and how a truly universal version manager.

bin_bash4• 3 years ago

that would be unrealistic at least for now. The plugins are all written in bash. You'd first need to rewrite enough plugins for it to be useful. Maybe at one point if rtx takes off I could look into having plugins be in an alternate form though.

happens• 3 years ago

Nice! I tried something similar at github.com/happenslol/qwer, I'll look at yours to see what you did differently :-)

codethief2• 3 years ago

Great to see someone is trying to provide a better alternative to asdf!

aliases and fuzzy-matching

What exactly does fuzzy matching mean? The only examples I see in your README are of the nature

  rtx install nodejs@20.0.0       Install a specific version number
  rtx install nodejs@20.0         Install a fuzzy version number

How is this different from doing specifying `nodejs@20.0.x` (which I guess asdf doesn't support)?

In any case, I hope that, if I try to install a package/plugin and it doesn't exist, it won't just try to install a plugin/package whose name is similar? :nervous_look.jpg:

bin_bash4• 3 years ago

What you described is fuzzy matching, basically just prefix matching.

asdf has a handful of commands that let you do prefix matching like `asdf install nodejs latest:20.0` but they're not supported in `.tool-versions` files. This basically just adds that functionality.

I didn't feel like requiring `latest:` was necessary. I felt it was clear that `nodejs 20.0` means `nodejs 20.0.*`. This is how npm has always worked.

ecnahc515• 3 years ago

This is awesome. Have you seen http://tea.xyz? Made by the creator of brew. I think it has a lot of interesting ideas, especially since it also has some of the same features as asdf/venv/rbenv/etc: managing multiple versions.

Figured I'd mention it, because I haven't tried tea yet, but I'm an avid asdf and brew user and I've been looking at asdf alternatives.

jdxcode7• 3 years ago

I haven't, thanks for sending. It's interesting that it hooks into the "not found" hook of the shell so you can run things like `node^18 --version`.

I also really like that it has really clean install output.

canadaduane• 3 years ago

Automatically installing whatever missing thing that you type as a command sounds like a terrible idea. Am I missing something?

jemmyw• 3 years ago

It looks interesting. I had been avoiding it because the last time I read about it the creator was going on about blockchains and the current maintainer of brew weighed in saying that brew is totally unassociated with that person so take any comparison with a grain of salt. I'll probably avoid tea until it gets a bit more stable though (if ever). These things take time, homebrew took quite awhile to be really great.

jordemort• 3 years ago

I actually view being written in shell as an advantage for asdf. That means I can just submodule it into my dotfiles and bootstrap my tools without extra steps, no matter where I end up.

sigmonsays• 3 years ago

Obligatory "Written in Rust " after everything written in rust is still funny to me.

sph• 3 years ago

I'd like to see the post-modern take on that phenomenon: take a Rust tool and port it to C89.

aliqot• 3 years ago

I'm game, which tool?

herewulf• 3 years ago

Well, I came here wondering why a Common Lisp library (module)[1] had been rewritten in Rust. Whoops.

[1] https://asdf.common-lisp.dev

jdxcode7• 3 years ago

I knew you wouldn’t be the only one so I made sure that “rtx” was not a popular lisp library. :)

wpyoga2• 3 years ago

I have been using asdf for over a year now, and I haven't seen `node -v` take so long. Actually, I used to use nvm and it was much slower than asdf, which was my main motivation to switch.

I'm wondering if the lack of slowness I've seen could be due to my simplistic setup: globally set nodejs version, single nodejs version installed, with other tools like git installed by the distro package manager. I also have golang and some other stuff like rclone installed using asdf, but not much more.

Did you also have a similar experience when initially installing node? Did you notice what triggered it or when did asdf start to become slow?

I like asdf and would like to avoid making it slow on my laptop.

Oh and can I safely assume that your home directory is on an SSD and not on an HDD?

jdxcode7• 3 years ago

I'm on an M1 MacBook Pro 16". The performance issues are well understood by the asdf team and the Github issue with most comments is on the topic: https://github.com/asdf-vm/asdf/issues/290

This was with a half dozen plugins and use_legacy_file disabled. Hardly an edge-case. Though I'll note in other tests it was around 200ms (which is the number I put in the README to be fair). @danfritz in this thread said his can take over 2 seconds!

Out of curiosity, what is the difference between `time node -v` and `time ~/.asdf/installs/nodejs/*/bin/node -v`? I've never seen it not add at least 100ms.

aorth• 3 years ago

For me, on a ThinkPad 14s with an SSD on Linux:

  $ time node -v
  v18.13.0
  node -v  0.05s user 0.02s system 94% cpu 0.073 total
  $ time ~/.asdf/installs/nodejs/18.13.0/bin/node -v
  v18.13.0
  ~/.asdf/installs/nodejs/18.13.0/bin/node -v  0.00s user 0.00s system 94% cpu 0.004 total

I've never noticed asdf being slow, so I'm surprised to see those extra ms on top when calling the shimmed node.

wpyoga2• 3 years ago

Apologies for the delay. Now I do notice a delay when using the asdf shim instead of running the binary directly. But it's nowhere near 900ms.

I'm on a T480s with i5 CPU and SSD:

  $ time node --version
  v18.13.0
  
  real 0m0.080s
  user 0m0.079s
  sys 0m0.027s

  $ time ~/.asdf/installs/nodejs/lts/bin/node --version
  v18.13.0
  
  real 0m0.009s
  user 0m0.006s
  sys 0m0.003s

codethief2• 3 years ago

As mentioned in my other comment already, it's great to see someone is trying to provide a better alternative to asdf!

From what I can tell by looking at the documentation of `rtx install`,

  rtx install                # installs all runtimes specified in .tool-versions for installed plugins

and of `rtx plugins install`,

  install a plugin

  note that rtx automatically can install plugins when you install a runtime

e.g.: `rtx install nodejs@18` will autoinstall the nodejs plugin

I take it there is still no way in rtx to simply install all plugins and runtimes listed in .tool-versions, without listing them one by one on the command line? Put differently, I would expect `rtx install` to install all runtimes specified in .tool-versions, including any plugins that are not installed yet.

Would this be something that you would consider adding? I'm asking because not being able to install all plugins mentioned in .tool-versions automatically has been my biggest gripe with asdf.

jdxcode7• 3 years ago

I actually had this as the default behavior of `rtx install` but it felt a bit wild for it to just go out and download a bunch of stuff. I feel like it’s nice to have a bit more control over which runtimes are and are not managed by rtx.

That said, I was thinking about this yesterday and that it’s currently painful to have to type out each plugin one by one. I added this ticket for making `plugins install` default to installing all plugins in the local tool-versions: https://github.com/jdxcode/rtx/issues/50

What do you think? Is that discoverable and useful enough? I could go a step further and add a `--plugins` argument to install or something.

jdxcode7• 3 years ago

I just added `rtx install --all` which installs all plugins and all runtimes. It's in v1.3.0

Show HN: Asdf Clone Written in Rust

Related Stories