One Year of Nushell

https://www.nushell.sh/blog/2020/08/23/year_of_nushell.html

by rainworld • 6 years ago

213 47 6 years ago

Nushell is the only shell I know of (other than fish) which uses threads. Threads are famously difficult in shells because of the fork/thread interaction.

How is Nushell able to use threads? For example how are subshells implemented?

vlmutolo• 6 years ago

Part of it could be due to using Rust as the implementation language. Rust has good, cross-platform abstractions over things like threads and processes. The language tends to make it harder to shoot yourself in the foot.

Would definitely like to hear from the authors of Nushell on how their language choice has held up over the course of the project.

ridiculous_fish2• 6 years ago

A related issue is leaking fds into child processes. For example, Go locks around fork() to prevent racing with creating new fds (e.g. from pipe) [1]. Rust doesn't have any synchronization between fork and creating fds; I wonder whether/how this is addressed in nushell.

1: https://golang.org/src/syscall/exec_unix.go?s=7064:7143#L19

Matthias247• 6 years ago

My guess it doesn't use fork, apart from spawning subprocesses use fork/exec combinations. The old fork idiom isn't exposed by Rusts standard libraries and also wouldn't be "safe" to use, since the basic expecatations of the language about how many things have access to a certain variable wouldn't hold up anymore.

laksdjfkasljdf• 6 years ago

reading only this post, is nushell just an implicit awk after each pipe, that already know the output format of a few common programs? or something more than that?

algorithmsRcool• 6 years ago

Think powershell, but built in rust, posix first and better at processing data(xml, csv, json, ...)

xfer2• 6 years ago

I looked at the book, currently it is nowhere close to powershell.

As far as i can tell no way to define any function/commands(you have to write plugins in another language, make an executable??). It supports pipeline, has a large amounts of built-in operators/commands(probably enough to be usable as a shell) and built-in parsers for a few formats(how does one write support for another data format?). Maybe in future it will address these questions.

emersonrsantos• 6 years ago

But Nushell is POSIX compliant?

eigenrick3• 6 years ago

It is quite a bit more in-depth than that.. It really is more of a structured data pipeline in a programming environment. It's more like Spark than Awk.

There are bulti-in methods like `parse` and `split`, `format` which can provide awk-like behavior. However, there are many other built-in methods which operate on tables, floats, vectors and integers.

I actually have a ticket in to make some of the parsing behavior a bit more "awk like" (https://github.com/nushell/nushell/issues/2375)

ChrisSD2• 6 years ago

It also turns out that getting a shell correct is a lot more experimentation and a lot less following what the textbooks say. In practice, things like ‘stdout’ and ‘stderr’ are used in a variety of ways, as are the exit codes output by programs. Our initial designs started with a stricter view of these ideas, and softened over time as we saw that the real world is a much fuzzier place than we realized.

Good point. Exit codes can be difficult. A non-zero error value may not be an error; it could indicate a type of success or be a warning or a result of a calculation or any number of things depending on the application.

wwright2• 6 years ago

At work we have a program that uses the exit code to encode three enums. Each digit of the decimal representation of the exit code corresponds to one of these enums.

It’s some of the most cursed code I’ve ever worked with.

(God bless anyone who ever used this program and got a set fault or sigint or sighup and therefore a bizarre result.)

etaioinshrdlu• 6 years ago

That is a very bad design. Especially since you have only one byte to work with.

What is the decimal range of values used?

samatman• 6 years ago

I was horrified by your post, because I've had to deal with load-bearing exit codes.

Then, I read the part about each digit of the decimal representation.

Now I may need to lie down for a spell.

adamch• 6 years ago

In a redesigned shell, ideally there would be a richer way to describe the result of a program. Maybe "return an integer and consult the manual to see what it means" isn't the best way forward.

Personally, I think Result types from Rust/Haskell/Swift make a lot of sense -- a program either succeeds with some output, or fails with an explanation of why. By default, pipes stop piping if one command fails.

moonchild• 6 years ago

Personally, I think Result types make a lot of sense -- a program either succeeds with some output, or fails with an explanation of why

Usually, this is what happens. Usually, the error-signalling return is accompanied by a message (posted to stderr).

By default, pipes stop piping if one command fails

  set -o pipefail

karl42• 6 years ago

Plan9 returns an error string instead of an error integer with the empty string signifying success. This seems like a simple and useful improvement to me.

franga2000• 6 years ago

In a redesigned shell

Correct me if I'm wrong, but aren't exit codes an OS-level thing? That would be one hell of a redesign!

Not that I wouldn't absolutely love to have rich/structured return values, but that's already hard to do in a single-language system - even ignoring backwards compatibility, doing this in a traditional OS would require a level of cooperation I don't think the industry is capable of.

majkinetor3• 6 years ago

In a redesigned shell, ideally there would be a richer way to describe the result of a program

And there is in PowerShell - you can return arbitrary complex object.

geophile2• 6 years ago

Marcel is another pipe-objects-instead-of-strings shell. It takes a different approach to shell/language unification, compared to nushell and other modern shells. Marcel exposes Python on the command line, e.g. to find files that changed in the last day:

    ls -fr | select (f: now() - f.mtime < days(1))

ls pipes File objects to the select, and that's a Python function inside the parens.

Scripting is done in Python, using marcel's shell commands as functions, by importing the module marcel.api. So the examle above in Python would be:

    from marcel.api import *

    for file in (ls(file=True, recursive=True) |
                 select(lambda f: now() - f.mtime < days(1))):

    print(file)

So it's the same shell command (ls ... | select ...), but expressed as Python functions, and the result is an iterator so that the command output can be used inside a for loop.

For more info: https://www.marceltheshell.org, https://github.com/geophile/marcel.

xfer2• 6 years ago

It's not a different approach though. It works exactly the same way in powershell(just the host language is different).

geophile2• 6 years ago

Thanks, I wasn't aware of that. I believe my comment still applies to nushell and a couple of others.

I like bash for simple commands, but hate the requirement to learn so many sublanguages for more sophisticated usage, (as in sed, awk, find, and others). I like Python for scripting, but the integration of shell commands is really not very good. I built marcel to fix both of these problems.

garciasn• 6 years ago

Is it just me or does the lack of line continuation (i.e. \) really ruin all of the beauty the rest of this shell provides?

svnpenn2• 6 years ago

My issue is that NuShell has line hinting, with no way to disable it

forbes4• 6 years ago

What is line hinting?

svnpenn2• 6 years ago

if you type a letter, it fills the rest of the line with what it thinks you want to type next

forbes4• 6 years ago

This seems very interesting. After a quick play I cannot find anything similar to 'less' which I use very frequently. Is there a nushell equivalent?

kbd2• 6 years ago

less is a program, why can’t you use it from nu?

forbes4• 6 years ago

If you use less with a filename, it will work. I normally pipe things to less which doesn't seem to work for me in nu. less shows an empty input.

This is possibly me just doing something wrong.

eigenrick3• 6 years ago

The equivalent is called `textview` which is a built-in which tries to be a bit more intelligent than `less` with regard to file-types.

The default viewer, which sort of runs at the end of every pipeline is called `autoview`.

Now.. autoview should be calling textview in the cases where the pipeline detects a large text value. It is possible that you're not exactly running a pipeline.

For instance: If you run:

    cat big.log

it is not running a pipeline, just an external command.

If you run:

    open big.log

 you're invoking nushell's mechanism for creating a pipeline sourced by the contents of big.log.  You should also notice that textview is invoked, because there should be a pager.

Alternatively, you can choose to create a pipeline by pipe-ing the output of cat big.log into something. If you pipe it to textview you should see the same result:

    cat big.log | textview

Hope that helps!

forbes4• 6 years ago

The command that I was trying to use was 'ls', which doesn't seem to play with textview when I try it out 'ls | textview' (just no output at all). textview does work for me with files, as in your example.

eigenrick3• 6 years ago

One of my favorite, unexpected features of nushell so far is `$it` the implicit "iteration" variable. It makes sense to have something like this in conjunction with the 'each' command, which makes it very easy to loop over lists or tables.

But $it is still useful in the absence of 'each'.

For instance:

dirname is a useful tool but you can't pipe to it (at least not the GNU version), it expects a string parameter. We can still easily plug it into nushell pipelines though, thanks to $it, which is this case is basically turning stdin into a string.

Since dirname won't let us do

    which nu | get path | dirname

we can instead do

    which nu | get path | dirname $it

If I wanted to do this in bash, I'd have to go through some annoying (but common) machinations in order to convert stdin into a text string to pass to dirname.

nitroll• 6 years ago

Like which nu | xargs dirname

One Year of Nushell

Related Stories