Fish – Update on the Rust port

https://github.com/fish-shell/fish-shell/discussions/10123

by rhim • 3 years ago

335 156 3 years ago

behnamoh• 3 years ago

one of the fastest shells out there will probably get even faster. ever since I switched from zsh to fish, my terminal productivity has gone up a lot.

bayindirh3• 3 years ago

I don't think C++ and Rust has that big of a performance difference. They are probably neck to neck with no clear winner.

LoganDark2• 3 years ago

The real difference is in compile-time abstractions. It's much easier to organize things in Rust, especially with a borrow checker. You can confidently remove overhead while being sure that it'll continue to be memory safe.

joshlk• 3 years ago

In Rust it’s easier to leverage external crates/libraries that are more optimised than rolling your own. This can lead to performance increases (I don’t know if fish is doing this)

andrewaylett2• 3 years ago

The advantages of Rust come when trying to do complex things with concurrency, and wanting to know that they're either going to be safe, or fail to compile.

Consider the now-classic "Fearless Concurrency" post about Stylo: https://blog.rust-lang.org/2017/11/14/Fearless-Concurrency-I...

Strictly, C++ pretty much has to be no slower than Rust. But it's certainly possible that a regular human might in practice write higher-performing parallel code when using Rust.

nick__m2• 3 years ago

I would not bet on this!

from the link :

  We have no real idea what the performance of the finished port is going to be. Early results are encouraging, but it is entirely possible this will be ~20% slower in some cases.

tubthumper8• 3 years ago

This part also seems relevant:

Fish also uses threads, for things like the autosuggestion and syntax highlighting, and we would like to have more "threads"/"concurrency"/"run-thing-in-backgroundness" (please insert the correct term here, I have never been good at these things). One large project has been to run multiple fish builtins and functions "at the same time", to enable things like backgrounding functions (ideally without using "subshells" because those are an annoying environment boundary that shows up in surprising places in other shells), and to simply be able to pipe two builtins into each other and have them actually process both ends of the pipe "simultaneously".
C++ offers few guarantees on what can be accessed from which thread. @ridiculousfish has been trying to crack this for years, and hasn't been confident enough in his solution. We want a tech stack that helps us here, and C++ doesn't.

The initial rewrite may be no faster, or possibly slower as stated, but long-term they'd be able to safely add more concurrency or parallelism that they weren't able to before.

fbdab103• 3 years ago

Is shell performance a limitation for some workflows? I naively think of it as some glue which shunts the work elsewhere. Maybe in the efficiency of pipe? I suppose those lunatics who write full programs in bash?

I will always take faster code, but that seems like an odd bulletpoint to highlight given the many quality of life improvements fish has to offer.

kstrauser4• 3 years ago

Kind of, yeah. When I used zsh, oh-my-zsh made the shell startup a lot slower, even though I wasn’t going crazy with the customizations. Opening a new terminal tab would take a couple seconds to get a prompt. My current fish setup is imperceptibly fast to start.

Those little delays add friction. Not an intolerable amount, sure, but it’s there. While I switched to fish for other reasons, that alone would keep me from going back now.

jonS90• 3 years ago

If it takes 5 seconds for your prompt to display or become interactive, yes.

As far as I'm aware, most of the other quality of life improvements can be reproduced in zsh with plugins. This performance bulletpoint makes me want to consider switching back to fish from zsh.

bch• 3 years ago

one of the fastest shells out there will probably get even faster […] my terminal productivity has gone up a lot.

Surely you’re not typing and managing jobs at a pace where C vs C++ vs Rust vs $lang matters… or I’m missing something about what fish is bringing to the table.

cube22224• 3 years ago

1. It's amazing that they're doing this as a gradual C++ to Rust rewrite, while keeping it working end-to-end, if I understand correctly.

2. It's amazing how quickly this is going.

3. If you haven't tried fish yet, make sure to do so! It's a really ergonomic shell and overall very pleasant to use, with good defaults (you don't have to customize it, even though you can, for a great experience). I've switched from bash a couple years ago and haven't looked back since.

Bonus: you won't have to google how to write a for loop in bash ever again (which I, writing them rarely, and them being unintuitive enough, had to do every single time)!

shmerl3• 3 years ago

Tip for some easy to remember Bash loop constructs:

Incremental index:

   for ((i = 0; i < 10; i++)); do echo $i; done

For iterating over an array:

   foo[0]=a
   foo[1]=b
   for i in ${foo[@]}; do echo $i; done

cube22224• 3 years ago

Just to illustrate for others, in fish that would be

  for i in (seq 10); echo $i; end

or usually written as

  for i in (seq 10)
    echo $i
  end

and that is already based on iteration, so for iterating over files, you'll similarly use

  for f in (ls); echo $f; end

and it works the same for arrays.

arp2424• 3 years ago

zsh makes this so much easier. This doesn't capture $i, but covers most use cases of "I want to run something more than once":

  repeat 10; echo "repeat me"

If you do need $i you can use a similar construct as bash, but more convenient:

  for ((i = 0; i < 10; i++)); echo $i

  for i in {0..10}; echo $i  # {n..m} loops also work in bash

  for ((i = 0; i < 10; i++)); { echo $i; echo $i }  # Multiple commands

Short loops are hugely helpful in interactive use because it's much less muckery for short one-liners (whether you should use them in scripts is probably a bit more controversial).

Also looping over arrays "just works" as you would expect without requiring special incantations:

  arr=(a b)
  for v in $arr; echo $v

---

Whether zsh or fish is better is a matter of taste, and arguably zsh has too many features, but IMHO bash is absolutely stuck as a "1989 ksh clone" (in more ways than one; it uses K&R C all over the place, still has asm hacks to make it run on 1980s versions of Xenix, and things like that).

darrenf• 3 years ago

For the first, I prefer range syntax:

    for i in {1..10}

stouset4• 3 years ago

Please please please always quote variable expansion. Just do it everywhere, every time.

    for i in “${foo[@]}”; do

madeofpalk• 3 years ago

“Just remember how to do it” isn’t really the problem.

It’s that the finer details of the syntax is sufficiently different from other things I do, and I don’t write shell scripts frequently enough to remember it.

mgdlbp2• 3 years ago

zsh, short loop:

    for i in {1..10}; <<<$i

powershell:

    foreach ($i in 1..10) { $i }

    1..10|%{$_}

xorcist2• 3 years ago

For iterating over an array:

Bash array syntax is error prone and hard to read, and the expansion of arrays still depend on the field separator and must fit in the environment.

Most of the time you should just rely on that field separator and do it the simple way:

  for i in "$f"; do echo $i; done

Much more obvious and more shell-like. That's how all bash functions process their parameters.

Set IFS only if you really need to. But in that case also consider something like xargs where you are not limited by environment, have null terminated fields, and offer parallelism.

Arrays are only useful when you need multidimensionality, at which point you should probably look at using data files and process with other tools such as sed. Or start looking at something like Perl or Python.

benj1112• 3 years ago

Can't speak for others but one issue is the "; do"

I can generally cobble together python, and it'll be syntactically correct. I just need to check libs.

With shell I need to stop and make sure my semicolons in a for loop are correct.

If you're a heavy user it probably isn't a problem but all the little warts just make it difficult for an occasional user to keep a good enough model in their head.

capableweb• 3 years ago

1. It's amazing that they're doing this as a gradual C++ to Rust rewrite, while keeping it working end-to-end, if I understand correctly.

Seems to me they're not doing it gradually at all.

Another thing:
We plan on not doing any partial-rust release.
That means we would be doing e.g. fish 4.0 as fully rust and zero C++, and I think, contrary to what we usually do that warrants a beta. (Ordinarily we've stopped doing betas and release candidates because they simply don't get any testing).
We also still want to do a 3.7.0 in-between release that is still purely C++, so we have a better jumping off point for platforms that can't handle the rust version. It would be 3.6.1 with some neat changes backported.

https://github.com/fish-shell/fish-shell/discussions/10123#d...

Cu3PO422• 3 years ago

It's a bit of both. They are not doing any mixed C++/Rust releases, but you can check out the source and build it in the current mixed state and it will produce a binary that contains parts from both languages and works as a Fish shell.

Animats• 3 years ago

The price of C++ compatibility is that it doesn't use Rust strings internally. It's all "wchar" and "WString", which are Microsoft C++ "wide character strings". This may be more of a Microsoft backwards compatibility issue than a C++ issue.

mb7733• 3 years ago

Seems to me they're not doing it gradually at all.

They are doing the release all-at-once, but developing gradually. From the original PR[1]:

This should be thought of as a "port" instead of a "rewrite" because we would not start from scratch; instead we would translate C++ to Rust, incrementally, module by module, in the span of one release. We'll use an FFI so the Rust and C++ bits can talk to each other until C++ is gone, and tests and CI keep passing at every commit.

[1] https://github.com/fish-shell/fish-shell/pull/9512

giancarlostoro• 3 years ago

I still rather write Bash scripts or just bust out Python or Perl, but I definitely prefer Fish for terminal usage.

frankjr• 3 years ago

More on the motivation behind the rewrite.

https://github.com/fish-shell/fish-shell/pull/9512#issuecomm...

silverlyra• 3 years ago

Wow, thank you for posting this. I’ve been kinda curious to try Fish over the years, but seeing this message from one of the maintainers – how mature and well-reasoned, how both charmingly joking and firmly assertive it is – now really gets me thinking that I’ve been missing out on using an excellent project.

acdha5• 3 years ago

I used bash heavily for years but decided I’d spent enough time maintaining dotfiles & working around quirks around the middle of the last decade. I looked at a few alternatives but since fish had everything I wanted builtin, I decided to try it for a while. After the first day, I decided the experiment was over and switched everywhere. It’s great software both for how much it does and how little it forces you to think about how it works as opposed to what you’re trying to do.

noobermin2• 3 years ago

It is kind of telling that they thought porting to c++17 is "too much of a hassle" but rewriting everything in a new language isn't! Sorry, you're never ever going to beat "we did it to chase the new shiny" which is the story across the dev world.

That said, I don't use fish and wish them the best

coldtea2• 3 years ago

It is kind of telling that they thought porting to c++17 is "too much of a hassle" but rewriting everything in a new language isn't!

"Too much of a hassle" can mean "too much of a hassle in absolute terms" or "Too much of a hassle compared to what you gain". And porting to c++17 is probably high on the latter...

trealira2• 3 years ago

They said that they thought it wasn't worth updating from C++11 to C++17. In their defense, C++17 adds only some incremental changes. There isn't much to port anyway.

Although I don't disagree with this:

Sorry, you're never ever going to beat "we did it to chase the new shiny" which is the story across the dev world.

I don't think that's the case here. Rust offers more advantages over C++11 than C++17 or C++20 do.

For people who don't want to find the relevant part of the original post, this is it:

We moved to C++11 in 2016 (some quick calculation shows that's 5 years after the standard was released), and we're still on C++11 because the pain of upgrading is larger than the promises of C++14/17.

3PS• 3 years ago

As someone who daily drives fish on ~4 different operating systems and loves it, I've been really excited about this rewrite and blown away by how fast it's been progressing given the size and complexity of the codebase. It will also somewhat lower the barrier to contributing for a lot of people, including myself.

Another thing I'm very excited about, which has received less attention, is the planned future fish rewrite to use UTF-8 instead of wchar_t (typically UTF-16). UCS2 and UTF-16 have been a plague on software and fortunately Rust makes working with UTF-8 a breeze.

cpeterso2• 3 years ago

wchar_t (typically UTF-16)

Complicating supporting multiple compilers in one code base, sizeof(wchar_t) is 2 bytes in MSVC (UTF-16) and 4 bytes in gcc (UTF-32).

ordu2• 3 years ago

MSVC isn't used on Unix, is it? It is probably not a concern for them, because they do not support Windows anyway.

esafak• 3 years ago

Why, was the C++ codebase becoming unmanageable?

theshrike79• 3 years ago

Comment by the creator: https://github.com/fish-shell/fish-shell/pull/9512#issuecomm...

tl;dr

  - string handling
  - threading
  - cmake issues

darthrupert• 3 years ago

Becoming?

DoctorDabadedoo2• 3 years ago

I was raised in C++ spaguetti code bases, I didn't see a loved and well groomed project until I became a man.

paradox4604• 3 years ago

Fish is such an underrated shell. They make some very strong, opinionated choices (like ditching POSIX compatibility), and the end result is an extremely nice shell. I've been using it for a bit over a decade now, and still love it.

Fish shell scripting is also extremely pleasant to write, it feels like an ergonomic bash.

There are a few warts, as with any project, but none of them have ever actually ever blocked me from just getting what I wanted done.

thowaway91234• 3 years ago

I love fish. Switch from ZSH+Prezto to just plain Fish and I'm not missing anything and it's way faster. All the servers have bash and I still write bash scripts, but for day-to-day terminal stuff it's Fish all the way for me.

asylteltine• 3 years ago

I’m glad rust is replacing legacy languages like C and C++. It’s time to stop using memory unsafe language ffs!

exxos2• 3 years ago

So, what new features do we get from that? Except that it doesn't work on Windows any more?

jenadine• 3 years ago

Never worked on Windows. The programming language has nothing to do with that anyway.

exxos2• 3 years ago

It does.

With Go, it would have worked out of the box on all the targets Go can support.

speed_spread2• 3 years ago

fish.exe works on Windows just fine if you install it as part of MSYS2, which I prefer anyway to the kludge that is WSL.

ianschmitz• 3 years ago

As someone who uses zsh + spaceship, what benefits would I gain moving to fish as my main shell? I don’t write many bash scripts so the scripting side is less of a concern.

tipsytoad• 3 years ago

Eh, you can probably get very similar to fish with zsh + loads of plugins, but fish has lots of niceties out of the box (syntax highlighting, autosuggestion based on your directory). I've been using fish + starship with just a fzf plugin and it's got everything I need.

Check it out, it's effective with very little config

flexagoon• 3 years ago

Speaking of starship, today I realized that I don't really need any of its features, I just like the way it looks. So I replicated it with pure fish by creating a simple function:

  function fish_prompt
      # Fedora Silverblue workaround
      set -l pwd (string replace /var/home /home $PWD)
  
      echo
      echo -s (set_color -o cyan)(prompt_pwd -D 3 $pwd) (set_color magenta)(fish_vcs_prompt)
      echo -ns (set_color green) "" (set_color normal) " "
  end

ilaksh• 3 years ago

For me it's really about the autocomplete.

DoctorDabadedoo2• 3 years ago

Tbh, little, mostly convenience (best metaphor I can give is installing Sublime Text vs manually installing your vim and your own personal configs). By default it supports a bunch of interesting things, autocomplete, iteractive filesystem navigation, theming (I used oh-my-fish in the past, not sure how it is now), etc.

I like it, I would love to run it as my main shell, but I just do too much scripting for my own good to hop on this wagon.

MrJohz• 3 years ago

If you've already set up your zsh as you like it, I don't think there's a huge amount of benefit. The big benefit of fish is that it's pretty much the ideal shell, but straight out of the box, without configuration. Autocomplete just works, navigating the history is easy, it completes commands from your history, choosing a shell prompt can be as simple as opening a browser, etc. You can customise it pretty much as deeply as you need to - just like zsh - but the defaults are good enough that you've got a really nice shell out of the box.

petepete• 3 years ago

I switched from zsh to fish a couple of years ago. My setups are very similar in capability but my fish config is about 10% the size. I get nearly everything I want for free and now only use one plugin, fzf.fish

acdha5• 3 years ago

My rationale for picking fish over zsh in 2017 was that it had everything I wanted included, as was quite noticeably faster. Since then I’ve spent almost no time maintaining shell configuration, which has been pleasant.

kstrauser4• 3 years ago

The first time I learned to write my own shell prompt by creating a file named “fish_prompt.fish” in the right directory, then writing a function in it named “fish_prompt” that echos a string to stdout, I was nearly ready to uninstall zsh. Using it ruined me on stuffing stuff into $PS1 and crossing my fingers.

I highly recommend everyone at least try it for a few days, if only to see what kinds of things you’d always taken granted are actually shell-specific. Worst case: you say “this isn’t for me” and go back to bash/zsh/whatever happy that you’re still using your favorite shell.

butshouldyou• 3 years ago

Speed!

fish does a lot out of the box that I needed plugins for in zsh. zsh's plugins are slow, so my shell startup was over three seconds. fish starts way faster, even with plugins.

rodrigodlu• 3 years ago

When I evaluated my transition from bash to something else over the years, I found fish to be more cost-effective to maintain. Many times, I only need to install the distro package to feel comfortable, instead of loading all my dotfiles and things like oh-my-zsh.

Out of the box, effortlessly, I have autocompletion for commands; puts tabs/indentation automatically whenever I start an ad-hoc multiline command inside a for/if statement; a color prompt; etc.

I use fisher for plugins and now have more functions in my dotfiles, but it has far fewer dependencies — like 3 or 4 plugins. When I tried zsh, I immediately added several plugins with oh-my-zsh to feel comfortable.

I'm evaluating zellij instead tmux for the same reason. Some extra batteries included for free, but staying simple enough to keep.

jquaint• 3 years ago

I like use fish + starship.rs. Which gives a pretty similar experience. Starship can actually share you prompt across different shells so I get the same experience in zsh and fish.

I find I stick to fish more because: 1. has more features that I like out of the box. zsh requires oh-my-zsh to get similar features and that's just another thing for me to install every time. 2. Help menu tab completion. Fish has many completions for common shell apps that will display descriptions from the help menu from just hitting tab. 3. Scripting is much better, while you might not use it much now it much now you don't have a hard corner to cut yourself on if you do need it at some point.

chrysoprace• 3 years ago

I've always been curious about Fish, but was turned off by it not being POSIX-compliant. Is there anybody who can share their experiences with issues and workarounds in this area?

22c• 3 years ago

POSIX compliance is practically a non-issue for fish as it's not intended to be a replacement for bash/shell scripts. It's intended to be an interactive shell (eg. your daily driver) and to that end, it's quite good.

The downside of this is that you do need to "context switch" a little, especially if you've been using bash/zsh for most of your life. I find it easy to differentiate between when I am using the shell as a user interface and when I'm using bash/sh as an interpreter, but for others that might not be as intuitive.

I've also found that since I've stopped using POSIX compliant shells as my daily driver, I tend to "promote" my scripts to a more general purpose language much sooner than I used to when nearly everything could be done in bash.

hiepph• 3 years ago

Is there anybody who can share their experiences with issues and workarounds in this area?

Just use your Bash/zsh to run Bash/zsh scripts: `bash script.sh`.

turned off by it not being POSIX-compliant.

I was like this when I first started using Fish. I spent some time to learn some different syntax to get away from the POSIX-compliance. Overall, I think the saner syntax of Fish is worth the effort. Now I use the terminal much happier.

Besides, 90% of the time I don't need to follow POSIX-compliant. Many scripts or extensions nowadays support Fish.

For enterprise use, I still stick to Bash/zsh. They're the standard.

_lvbh• 3 years ago

Been using fish for a year. Although I don’t like how they break compatibility with standards so that some scripts don’t work (gvm, nvm, etc), it has all the features I would usually set up manually with zsh. After trying to configure zsh a few different machines, I gave up and went for default fish on all of them.

It’s already working so well, not sure what there is to gain by porting to Rust. Security? If something can maliciously tamper with fish, it’s probably already on your system or am I missing something?

paradox4604• 3 years ago

They have a variety of reasons to move to rust, as outlined in their original rust discussion[1]. Mostly around finding other contributors, and adding an async/parallel mode they're comfortable with.

[1] https://github.com/fish-shell/fish-shell/pull/9512

ochronus3• 3 years ago

Oh! I missed this, what's the original reason for doing the move? Any insight on this?

ochronus3• 3 years ago

Hm, why the downvote? I'm genuinely interested.

Vorh• 3 years ago

Presumably because the second to top comment links this issue, which answers your question: https://github.com/fish-shell/fish-shell/pull/9512#issuecomm...

Fish – Update on the Rust port

Related Stories