NGS: Next Generation Unix Shell

by yoodit • 10 years ago

151 199 10 years ago

IMHO the world does not need another Shell language. Please not... it's even sometimes complicated with ZSH & FISH today. A shell which behaves totally different will get me totally different problems than nowadays... no thanks. I know my "enemy and friend" Bash and it's a good thing I get it nearly everywhere.

philjackson• 10 years ago

I don't think there's enough that needs fixing in the current shells/terminals that require a new paradigm at the moment. These things seem to come and go without much adoption.

Sorry, bit negative of me - it's awesome that people are trying.

voidz2• 10 years ago

A shell is an application that provides system interaction to users.

What I see is a void. There is no good language for system tasks (and no good shell). What's near this void is outdated shells on one hand and generic (non-DSL) programming languages on the other. Both are being (ab)used for system tasks.

Aside from the, in my opinion, abhorrent word "outdated" (why are old things considered bad just because of their age?), the comment about there being "no good shell" is something I disagree with very much. Personally I'm really fond of zsh and its manual pages are a treasure trove. But, lest this digress into a flamewar about shells or into a non-productive discussion about preferences, the point I would rather make is that the author provides a different thing than, as it seems to me, qualifies as a shell.

For example, the open issue mentioned in the README is:

Open issue: how to deal with a command that requires interaction.

As a Linux system administrator I am happy with the tools I have, and initiatives such as this seem to digress into the realm of the programmer. Sysadmins and programmers (a.k.a. developers - though I'd consider myself a developer too but not a programmer) tend to have very different perspectives about how to get an application onto the environment on which it runs, within the greater system of servers and networks.

In that sense, Docker and such mostly seem to get a preferential treatment from programmers, while sysadmins (again, as it seems to me) tend to dislike these kinds of abstractions. And I often get the impression that developers haven't got as much appreciation for sysadmins and what they do, as vice versa, but this could be false, or even more likely is that this is true for some and false for other cases. OMMV.

NGS might be a very useful project, I'm not at all negative about the project itself. It just seems more useful to programmers than to people like me who enjoy nothing more than to interact with their command line interfaces a.k.a. shells.

CJefferson4• 10 years ago

I'm suprised you can't think of problems you currently have with shells. Here is a few of mine:

* Once I've started a program, and easy way of sending it to the background if it is taking a while, which sends it's output to some buffer I can refer to later, rather than continuing to spew it all over the screen.

* While we are at it, stop spewing the output of multiple programs over the screen, under any circumstances.

* Have some simple, easy to follow rules which let me work on files that have spaces in their names, without having to remember the various commands with various special cases (like -print0).

MereInterest• 10 years ago

For the first two issues, I use tmux. If something is long-running, I open a new shell to continue working. If I want to run multiple related commands, I can open many shells and have tmux send my keystrokes to all of them.

ZenoArrow2• 10 years ago

Those can all be seen as tweaks to existing shells. With regards to...

"* Have some simple, easy to follow rules which let me work on files that have spaces in their names, without having to remember the various commands with various special cases (like -print0)."

I'm not sure I understand what's hard about files with spaces in their names. I'd say it's easy enough to work with such filepaths by using tab autocomplete when using the shell interactively, and quote marks when writing shell scripts. Can you give an example of where these approaches wouldn't work?

voidz2• 10 years ago

Thanks to tmux, I don't have a problem with multiple interactive commands and their outputs. If I need to do something new, I open another shell on a new pane. Not that other solutions aren't welcome, but this works for me.

If I don't want output at all, the &>/dev/null output redirection works, or file descriptor redirection, for which zsh has useful methods.

Spaces in filenames are easily solved by quoting filenames or variables. Again, zsh provides a myriad of ways to make this a trivial issue.

My main point was to argue what qualifies as to being called 'shell'.

txutxu14• 10 years ago

issue 1) sounds like you miss nohup, sreen or tmux in front of that long running program.

issue 2) usually I don't have one terminal session with multiple programs doing output at the same time, this solves this issue.

Just use a new xterm, a new terminal tab, a new screen tab for programs that are going to produce output and stay running.

issue 3) I think you refer to files with other special characters, because spaces on filenames do not need -print0 at all, simply proper quoting in the code will do.

Regarding issue 3, there is a lot of caveats when coding shell; even file names starting with a dash (-) which are interpreted as options by tools external to the shell (which fortunately support a double dash (--) to stop processing parameters).

dumbguy• 10 years ago

Just use tmux. It'll take a day to figure out, and then all your problems go away. Well, save the spaces, that's never going to happen.

Intermernet• 10 years ago

I'd personally like a way to redirect the output of a running process (either `bg`d or `disown`ed) to a log file. The only way I've come across are horrible hacks using gdb[1] or strace.

This should be easier. Anyone know of any better ways? I'd like to be able to do something like `proclog <pid> <stdout output file> <stderr output file>`.

[1] http://stackoverflow.com/questions/593724/redirect-stderr-st...

cmiller1• 10 years ago

Open issue: how to deal with a command that requires interaction.

Yet this problem is solved very simply in bash and other shells by simply allowing you to explicitly tell it when to run a command in the background and when not to using the & command.

kkirsche2• 10 years ago

By and large I agree. As a programmer, I generally feel like the sysadmins which I work with "hide" how they do things and aren't transparent enough about their role for me to know why I should respect them. I've seen some do amazing things and then I recognize the difficulty but I don't feel their is enough collaboration between the two sides

voaie2• 10 years ago

The problem with outdated shells looks pretty clear: they were made with one kind of tasks in mind but are used for other, bigger and more complex tasks. Such scripts usually look as a fight against the language and working around it much more than using it to solve the problem.

Mind you,

Create a shell (in that language) that is up to date with today's tasks - cloud, remote execution on a group of hosts.

I'd rather create another DSL/tool to solve it.

Two languages actually.
Current-shells-like but simplified (called "commands"), $(...) syntax
Scripting language, "normal" programming language (called "code"), {...} syntax

Good luck.

The scripts that I've seen (and written in Python and Ruby) look too verbose and show unnecessary effort. Such scripts do not look an optimal solution (at the very least).

Did you check scsh? (I don't use it.) http://scsh.net/docu/html/man-Z-H-3.html

ilyash9• 10 years ago

I see now. The parenthesis don't feel like a convenient syntax.

voaie2• 10 years ago

Scheme could be a good language (for fans?) if it is not used interactively.

spc4764• 10 years ago

The QNX shell could easily do remote execution. I worked at a company in the mid-90s (porting Unix software to QNX). My machine had a modem on it, and my boss (the owner of the company) would use it from his computer to dial out---all he had to do was reference the modem device on my computer at his command line.

In fact, you could run a program on a local machine referencing a file on a second machine, pipe the output to a program on a third machine and have that redirect the output to a file on a fourth machine, all from the command line. I don't recall the exact syntax, but it was something like:

    cat @2/path/to/large/file | @3/bin/grep 'foo' >@4/tmp/output

(Individual hosts were numbered on a QNX ethernet based network, but the default permmissions were Unix like). All of this was a consequence of the message-based QNX operating system (whether messages were delivered locally or remotely was invisible to programs) but I don't see why something like can't be done today.

dmytrish2• 10 years ago

This is a quite neat example, but it looks like the QNX shell mixed several concerns about how do you connect to these machines, how do you secure the communication, etc.

    $ ssh @2 "cat /path/to/large/file" | ssh @3 "/bin/grep 'foo'" | ssh @4 "cat >/tmp/output"

is rather clunky, but separates those concerns well.

gcr• 10 years ago

Looks quite interesting! Some feedback:

Yes, a new paradigm for interacting with text commands would be wonderful. But what would that look like?

One strong advantage of the current system is that the input matches the interface. Everything is text, so it all can be typed with the keyboard. But if this shell "displays structured results as real fing structures (YAML, JSON, ...)", that may not be true. If the shell spits out a table, is it a CSV table? A YAML / JSON nested array? How is the user supposed to edit it? The interface isn't clear, and any layers at all are going to take some thought to be able to compose naturally with other tools. (The author addresses part of this with "All operations made via a UI, including mouse operations in GUI must have and display textual representation, allowing to copy / paste / save to a file / send to friend.")

Another thought: from the proposal in the README, it isn't super clear which tasks the author feels the shell ought to handle versus which tasks should be handled by some other toolkit. For example, the "Manage multiple servers at once" subsystem is handled by other devops tools like Ansible or Chef at the "deploy an application" layer and tools like Tmux or 'clusterssh' at the "send input to multiple processes in many screens at once" layer. Another example: the author proposes "Actions on objects that are on screen. Think right click / context menu.", but since these actions must require cooperation by the program being invoked, it's not clear whether this serves the purpose better than simply wrapping that program's operations in some GUI toolkit.

This could be pretty interesting, but I wish the author would formalize some of this thinking into a concrete standard. Having too big of a vision without some set boundaries seems to be hurting the focus of this promising project.

ZenoArrow2• 10 years ago

"One strong advantage of the current system is that the input matches the interface. Everything is text, so it all can be typed with the keyboard. But if this shell "displays structured results as real fing structures (YAML, JSON, ...)", that may not be true. If the shell spits out a table, is it a CSV table? A YAML / JSON nested array? How is the user supposed to edit it? The interface isn't clear, and any layers at all are going to take some thought to be able to compose naturally with other tools. (The author addresses part of this with "All operations made via a UI, including mouse operations in GUI must have and display textual representation, allowing to copy / paste / save to a file / send to friend.")"

In PowerShell, all data is stored as .NET objects. Using this object-oriented approach enables the sort of flexibility I believe you're looking for. Perhaps something similar could be developed for Linux. There has been some activity in this area:

https://github.com/Pash-Project/Pash

https://blogs.msdn.microsoft.com/powershell/2015/05/05/power...

http://www.forbes.com/sites/justinwarren/2016/03/08/is-micro...

ilyash9• 10 years ago

Manage multiple servers at once

IMHO will be much better if integrated into the shell. For example it will allow easy access to a shell variables with lists of hosts on which the command succeeded or failed.

akatechis• 10 years ago

Lots of assertions about how "bad" our current shells are, but no substantiation. I'm a developer, and I know just enough shell to write a simple deploy script or two, but reading this, I'm not convinced that what we really need is new shells and scripting languages.

ilyash9• 10 years ago

Simple example: using current shells it is not convenient to work with API call results which are structured data. Yes, there is jq but if the shell had data structures it would be much better, wouldn't it?

wicket3• 10 years ago

On UNIX, the "API" is basically that everything is a file and programs are written to handle text streams. After all these years, the shell is still perfect for this.

It sounds like you want create a new language and shell in order to have something similar to Windows PowerShell which interacts .NET objects using its API. This does not fit into the Unix "API" described above. This isn't a "Next Generation UNIX Shell" but an alternative shell that suits a certain group of users who don't really understand UNIX.

heydonovan• 10 years ago

I'm sure we can all agree that the current state of shells needs some work, but I don't think inventing a new one is the right solution. I'm a huge fan of the fish shell, but in the real world, it never seems to be installed across the farm, and convincing the older SysAdmins to install it is more trouble than it's worth.

We should be focusing on saner bash defaults, since it's the most common shell in use. We shouldn't have to remember these large list of gotchas, and pitfalls that shouldn't be there in the first place. A few people out there are trying to recreate common utilities such as ls or cp, and while I welcome the change, I feel it should be part of the actual gnu coreutils package, and not a new project: https://bsago.me/exa/

txutxu14• 10 years ago

If it's hard to try to convince them about fish... try to convince them to enter commands in a browser over a web stack, as a "shell".

If they are conscious about the parts involved and the security repercussion, maybe they will say the same: "no".

On the other side, technology is not advanced only by older SysAdmins, and it's nice to see new projects, experiments and developments.

jandrese• 10 years ago

try to convince them to enter commands in a browser over a web stack, as a "shell".

What? I had never heard of fish before and this is making me nop out pretty quickly.

nailer5• 10 years ago

That's because fish is a better version of existing shells. It's not significantly different to throw away what we have. Sane syntax and proper colors and better autocomplete make it better than bash, but not enough to throw away the network effects of bash. Think of hg trying to compete with git - it might be better, but it's not better enough.

As a shell person (I'm the maintainer of https://certsimple.com/rosetta-stone) NGS offers:

1. non blocking

2. structured results

Doing both of those - if NGS can do it - is a MASSIVE shift over existing Unix shells. Think git replacing svn.

The POC for #1 looks really good: https://www.youtube.com/watch?v=T5Bpu4thVNo

lfowles• 10 years ago

Personally, I use fish as my interactive shell. Bash still exists in the background for purposes of _the whole world runs on it_.

oblio4• 10 years ago

    We should be focusing on saner bash defaults, since it's the most common shell in use.

That kind of work is not glamorous and highly controversial. Ideally there should be some sort of cross distro/OS working group (Debian/Ubuntu, Red Hat, SUSE, FreeBSD, NetBSD, OpenBSD, Mac OS X, etc.), similar to the working groups that standardize the web, where such proposals can be made, voted on and adopted.

For example there should be no reason in 2016 why a sane solution can't be found for having almost every bash installation recognize all keys on the keyboard (i.e. arrow keys wouldn't produce ^A sequences or similar).

Another thing that would be sorely needed but would involve a much higher volume of work would be a template, at least for all GNU utils, which they use to define they options, parameters, arguments, whatever. And by "template" I mean library, actual working code they could include and configure.

Instead of each shell (bash, zsh) having to come with a million small scripts that configure auto completion, these shells could just query the standard-compliant tool for its usage and would receive a standardized reply with everything. Powershell has something like this and it is a great idea.

gioele• 10 years ago

That kind of work is not glamorous and highly controversial. Ideally there should be some sort of cross distro/OS working group (Debian/Ubuntu, Red Hat, SUSE, FreeBSD, NetBSD, OpenBSD, Mac OS X, etc.), similar to the working groups that standardize the web, where such proposals can be made, voted on and adopted.

Like the Austin group, the technical committee that constantly releases updates to POSIX and the Single Unix Specification? http://www.opengroup.org/austin/ http://pubs.opengroup.org/onlinepubs/9699919799/

iheartmemcache2• 10 years ago

I've been using -nix for almost 20 years. I've used everything from ksh on SunOS 2.6 (Solaris?? what's that) to oh-my-zsh (for 3 years, before happily graduating to oh-my-fish). I grew up on Slackware 6 waiting hours and hours for a 2.2 kernel build to finish. In high school, FreeBSD 4.3 kernel mods took up more time than booze and women. My 3 year puppy love for oh-my-zsh dimmed as I transitioned to a more sane, less emotional woman - oh-my-fish.

I hate to admit it, but Microsoft just got it right with Powershell. Standardizing the convention with an intuitive Verb-Noun, the out of the box documentation with -examples, -full, etc goes into so much detail that if everyone used it, there wouldn't be any dumb "How does I move directory??" questions on Stackoverflow (well, there'd be less at least).

Don't know a command? Show-Commands and type in 'network' to see what's available. Don't see it there? Go get a Powershell script (often offered by vendors like VMware and Citrix, making devop lives easier). I remember spending _weeks_ trying to get my dual-boot machine (FBSD / Win2k) Cygwin/msys setup to work well on my P3 600 at 12 or 13. Now I can just Feature install Bash and get a native binary. Let me say that again -- now, I can Feature install Bash on Windows jeeebus. QT is LGPL. CLR is open source. Visual Studio Community (basically Professional) is free (unless you need historical debugging then Ultimate's going to cost a lot). Satya + Meijers + Hanselman et al have made favor Microsoft so hard, despite being an active donor of the FSF.

That GNUutils-with-autocomplete is real trivial to write, but since it's already in fish, I see no need to right it. :smug: [In all seriousness, I agree with you re: standards. Plan 9's interface standards semi-addressed that. But le sigh such is life.]

GunboatDiplomat2• 10 years ago

I have to disagree with the idea that everything should be standardized. What's the point of different distros if everything is just going to be the same? Might as well just tell everyone, "Ok, we're all going to use CentOS."

StreakyCobra2• 10 years ago

Side note: https://github.com/supercrabtree/k is a ZSH plugin in the same register that `exa`.

xomateix• 10 years ago

Relevant xkcd: https://xkcd.com/927/

js22• 10 years ago

older SysAdmins

I hope you aren't referring to their age, in which case consider using a different adjective: experienced, seasoned, wiser, etc. Your colleagues probably don't want to install fish shell for reasons besides how old they are.

gervase• 10 years ago

Or, if you disagree with their opinion: hidebound, closeminded, timorous, etc.

I think the parent was going for something indicating disagreement, so it seems unlikely that your "positive" adjectives are what they were looking for.

kkirsche2• 10 years ago

This to me makes too many assumptions. I'm sorry but experience doesn't mean wise. And in many cases experience can mean complacent or scared of change.

chriswarbo2• 10 years ago

We should be focusing on saner bash defaults, since it's the most common shell in use.

Bash already provides two "modes": when called as `sh` it is more POSIX-compliant, when called as `bash` it allows more bash-specific features and syntax.

Maybe `bash` invocation can remain for backwards compatibility, and a new mode be added (e.g. `sash` for "sane shell") which can be used for breaking changes in the name of safety and sanity.

This opt-in usage would avoid breaking existing scripts; whilst being bundled as part of bash would mitigate some of the chicken-and-egg and paradox of choice. Over time, `/usr/bin/env sash` will appear in more and more guides/tutorials/etc. whilst those using `/usr/bin/env bash` will continue to work, but look a little clunky.

Of course, making such a "sane mode" as part of Bash would make it more high-profile and cause much more NIMBY, politics and bike-shedding than as a standalone project; but if it has a clear, consistent "mission statement" (e.g. "safety over convenience", "consistency over edge-cases", etc.), then maybe it could avoid too much scope creep and benefit from input by lots of very knowledgable experts.

stephenmm• 10 years ago

The new name has to be dcbash or maybe dcsh but dcbash is more fun.

donatj• 10 years ago

I find ZSH a much more reasonable alternative than Fish. Comes preinstalled on many Linux distributions as well as OS X, and can be configured with plugins to do most everything Fish does.

naasking• 10 years ago

I'm sure we can all agree that the current state of shells needs some work, but I don't think inventing a new one is the right solution.

Disagree. A shell is a domain-specific programming language, and all the existing such languages are terrible. We'd be better off with a properly designed language, like a shell language based on OCaml: Caml-shcaml [1].

[1] http://users.eecs.northwestern.edu/~jesse/pubs/caml-shcaml/

lomnakkus• 10 years ago

We should be focusing on saner bash defaults

Yes! A huge part of the fragility of shell scripting is the fact that there are a billion options which change crucial semantics of how the language works.

Just pick some goddamned sane option values and disallow changing them!

There's a reason you don't see this kind of nonsense in "real" programming languages: It plays absolute havoc with reasoning about what will happen when you run a script. (You could easily miss the fact that the shebang line does/doesn't contain "-e" or that option to error-on-unset-variables is/isn't set, etc.)

pjmlp4• 10 years ago

For me a next generation UNIX shell needs to catch up with what REPL environment in Lisp Machines, Interlisp-D, Mesa/Cedar, Smalltalk, Oberon(-2), AOS features and capabilities.

For the young HNers, think having something like Swift Playgrounds or IPython as your shell, while having full access to the OS API without relying on external programming languages.

Otherwise the next generation prefix isn't worth mentioning.

jordigh• 10 years ago

Something like TempleOS, written in Holy C?

gravypod3• 10 years ago

Terry really did something cool there with his JIT Holy C (or C+ before the TempleOS thing). I'd definitely like to see something like that ported to other platforms.

cm3• 10 years ago

And arguably you'd have to call it last-generation as well, if you recall how long ago those innovations happened but never made it into mainstream. That would label current shells as dark age of tty :). I still cannot accept the fact that we use tty emulators.

pjmlp4• 10 years ago

For me PowerShell feels a bit more closer to that model, but it is still text based, no inline graphics or data structures output, and the syntax could surely be improved.

StreakyCobra2• 10 years ago

As you said, something like IPython: https://ipython.org/ipython-doc/3/interactive/shell.html

mturmon• 10 years ago

Smart comment.

The UNIX shell and core-util model of "everything is text", and line-by-line output and tools for line-by-line data manipulation, goes very far, much farther than you'd think, but is limited. Everyone has internalized those limitations.

It reminds me of MATLAB: everything is a matrix (another standard, versatile, tabular data structure), and many language constructs exist to manipulate this common data structure.

andrey_utkin• 10 years ago

So, this project starts about shell, then I see it also aims to implement full-blown terminal emulator and a lot of new very ambitious features, and then it invites terminal-based apps to use new capabilities of it, so it would be best usable only if the whole ecosystem of compatible apps appear.

Unfortulately, I'm pessimistic.

Also, would like to comment on project's README. Not meaning to be harsh.

The shells never caught up.

Wrong. To prove this, it is enought to see that almost none of "features" is new in comparsion with traditional terminals.

Not to block, allow typing next commands even if previous command(s) still run(s).

Got a plenty of ways to handle that with shell. Backgrounding, GNU Screen, Tmux.

Open issue: how to deal with a command that requires interaction.

Oh yeah, you've invented a thing and don't know how to use it with real-world apps. Happens.

Provide good feedback. In GUI for example, this can be green / red icon

Enhanced setting of shell's "PS1" variable gives you good indication of exit code of last app (color + text + whatever, you may even use audio if you want). I'm using it for years.

All operations made via a UI, including mouse operations in GUI must have and display textual representation, allowing to copy / paste / save to a file / send to friend.

Sure, why not, but all this is possible with shells. Both native consoles (with gpm) and X11 terminal apps, and also Tmux has its own possibilities to handle clipboard.

Different UI modules must exist

Lots of people was playing around with shells via IRC or XMPP ages ago.

Web

Good luck handling keystrokes with tricky keycodes, and preserving all other standard terminal features. BTW I thought this project aims another shell, but this sounds like requiring another terminal emulator. I'd be glad to hear we have powerful web-based terminal emulator, but this feels very laborous.

allow multiple users to collaborate

Screen/Tmux.

some rw, some ro

Seems lacking in tmux at the moment, but possible to implement on its own, without reinventing everything else.

Most of the data dealt with is tables. List of files, list of instances in a cloud, list of load balancers. Amazing that none of current shell tools (I heard of) don't treat the data as such

It is discouraged to parse textual output of shell programs unless app clearly guarantees unchanging parsable/consumable format (with specific options you use). Quite often there's another, more reliable way to mine information than parsing of output of general-purpose program.

Allow editing it and saving to file

some_program | vim -

Then, in editor, edit and save. Done. If you add too many specific usecases into the core, you end up with bloated interface.

jerf• 10 years ago

"So, this project starts about shell, then I see it also aims to implement full-blown terminal emulator and a lot of new very ambitious features, and then it invites terminal-based apps to use new capabilities of it, so it would be best usable only if the whole ecosystem of compatible apps appear."

This is actually my only objection and the only feedback I'd make to the author. Contra some of the other posters here, I fully agree shell is a mess and that the basic root cause of the mess is that you can not straddle interactive use and programmatic use with one language. But something a bit smaller needs to be bit off.

I'd also suggest that most shells have already got the interactive use case sufficiently covered and that the low-hanging fruit is in the command case. Perl is perhaps the best language that functions as a programming-shell replacement, but using it for a while reveals it still has many significant issues in that use case. Even with modules, it pipes surprisingly poorly. It tends to make stream processing harder than it needs to be, unless "one line at a time" works for you. It's got a lot of nasty syntax that is arguably optimized for big program use cases that doesn't make sense when replacing shell scripts. When being used for shell programming, it basically shares C's abject failures when it comes to error handling, making it range between bizarrely difficult (I have to left-shift the result of system to get the actual exit code?) to almost-impossible-to-remember (backticks) to get errors properly handled. It has no concurrency story or any particular "run on multiple targets" ability (though I confess I don't know what that looks like necessarily; in my head it rather becomes puppet or ansible fairly quickly).

There's room here, but it's going to be an uphill battle.

JdeBP4• 10 years ago

[...] shell is a mess and [...] the basic root cause of the mess is that you can not straddle interactive use and programmatic use with one language.

Note that some of the world has already taken this notion on board long since. About a decade ago, Debian Linux and Ubuntu Linux swapped out /bin/sh so that it was no longer the Bourne Again shell. Nowadays, as a result of this, one regularly finds Linux systems where programmatic shell scripts are interpreted by something like the Debian Almquist or Debian Policy-Ordinary (posh) shells and interactive login session shell work is the domain of the Bourne Again, Korn, Z, or other shells.

And of course, having a different "better" shell for interactive use was the reported rationale for the C shell.

yoodenvranx3• 10 years ago

Something I would LOVE to see (not only in a shell but in all tools) is some kind of "project mode".

- In Chrome/Firefox I'd like to have a mode for recreational browsing, one for research of webdesign and one for fitness/health stuff. This basically means that I want to click at a browser window and say "This is now my fitness window. Please remember all the open tabs". And then there is a list of open sesions/projects where I can switch between those views/instances. I'd like to be able to close my fitness-window (without losing my fitness tabs) and then re-open the same fitness-window on the next day.

- For Dolphin/other_filemanager I'd like to see a "normal" mode for everyday browsing, a mode with large icons for sorting pictures and some mode for working with Python scripts in which the nautilus terminal is always open

- for Sublime I want a scratch window where I open and edit random small files, one for my html coding and one for my Python coding

Currently for almost every program we only either have one "session" or some really specific project settings (like Visual Studio) but I'd like to see something in between.

For example I have a lot of HTML files open in Sublime and I need to edit some unrelated config file. If I just open this config file it will open in the same window as the html files. What I'd like to have is a way to say via comment line or context menu "open this file in scratch pad mode".

The same for Dolphin. If I have one instance of Dolphin open for picture management with large icons and I have to do some Python file editing I'd like to have a simple way of saying "Open Dolphin in python mode".

Or sometimes I have a random link. I want to open that link in my "random browsing" instance of Chrome and not in the window which has currently all my fitness tabs open.

And the same is true for video players, music player, ebook programs, ...

mdrzn• 10 years ago

You should try Session Buddy for Chrome: it does just what you asked.

sksksk• 10 years ago

Chrome has this feature already, with the concept of Persons. I have a "Work" person and a "Home" person, which gives me a different set of cookies, extensions, pinned tabs etc...

yoodenvranx3• 10 years ago

I know about persons but I'd like to share my cookies and other stuff between all my "projects". I tried to use it but it is too separated for my uses.

azdle• 10 years ago

- In Chrome/Firefox I'd like to have a mode for recreational browsing, one for research of webdesign and one for fitness/health stuff. This basically means that I want to click at a browser window and say "This is now my fitness window. Please remember all the open tabs". And then there is a list of open sesions/projects where I can switch between those views/instances. I'd like to be able to close my fitness-window (without losing my fitness tabs) and then re-open the same fitness-window on the next day.

Sounds like you're asking for 'Tab Groups'. It used to be a feature built into Firefox, but recently got removed due to lack of use, it is however still available as an add-on (mostly using the existing code that was removed from Firefox itself afaik):

https://addons.mozilla.org/en-US/firefox/addon/tab-groups-pa...

yoodenvranx3• 10 years ago

I used to love and use Tab Groups but it was always amazingly slow on my PC (old-ish Celeron 2.6 Ghz)

JadeNB• 10 years ago

Firefox's Session Manager (http://sessionmanager.mozdev.org) does almost exactly what you want.

wiremine• 10 years ago

The github page links to a screencast which I found more helpful than the github writeup itself:

https://www.youtube.com/watch?v=T5Bpu4thVNo

anfroid5552• 10 years ago

But why nodejs?? There are a lots better languages to base this off ove.

anfroid5552• 10 years ago

But why nodejs?? There are a lots better languages to base this off of.

nailer5• 10 years ago

Probably because JavaScript is the worlds most popular programming language, it's portable, has the largest package manager, supports the shell's non blocking requirement, and the current version (ES6) has a better stdlib than previous versions.

tacos3• 10 years ago

You're wrong on about three of those statements, but note that the same hand wave could have been made for Perl ten years ago. That would've been a poor choice then for the same reason Javascript is a poor choice now.

ilyash9• 10 years ago

NodeJS was first throw-away implementation. Then Lisp. Now it's in C and looks like it will stay this way.

lottin• 10 years ago

Just because something has been around for a long time it doesn't mean that it's outdated. On the contrary, it means that it's stood the test of time, which is usually a sign of a good design.

ilyash9• 10 years ago

Sorry, by outdated I meant not a good fit for today's problems.

CaptSpify4• 10 years ago

Personally, I don't see how today's problems are that much different. I agree we should always be looking for better solutions, but IMO, it seems better to adjust our current tools than to rewrite them.

amelius• 10 years ago

I suppose all of the behavior should be highly customizable. It would be nice if plugins could be written in any language. For example, I expect special CLI editor extensions for editing JSON input to programs (with syntax highlighting if desired). Also, I expect specific key-bindings (for e.g. vi/emacs modes) to be pluggable.

ilyash9• 10 years ago

I encourage you and others to open issues or make a pull request with updated README

pmlnr• 10 years ago

Actually... bash is a pretty solid language since version 4, just no one takes time to learn it.

chungy• 10 years ago

Might you elaborate on what version 4 does new/differently?

Adaptive• 10 years ago

associative arrays are one of the biggest v4 advantages

s_kilk• 10 years ago

Having read through the readme, I think some of the issues outlined could be solved by having a cell-based model like jupyter/ipython.

Basically a command (input) and its corresponding output are tied together as a "cell". The UI could be very like the ipython notebook, where you have a text box for command input and then above it you have a chronological list view of cells. You would also have a good keyboard-shortcut language for navigating back and forward in the cell history, so you could, say, go back and focus a cell which has been running for a while and is now prompting for input.

(Edit: imagine a key sequence like `Alt-b, Alt-b, Ctrl-Enter` meaning "go back two cells and give text focus to that cells input field.". In reality you'd probably want a command language that's more vim-like.)

The cell model would also allow you to truncate output, minimize or maximise the cell display, and a bunch of other UI tricks I can barely think of right now.

Basically each cell would be a little parallel shell of it's own.

If this sounds even remotely compelling, I wouldn't mind writing up a more formal proposal and submitting it as an issue on the repo.

dustingetz• 10 years ago

If processes were referentially transparent functions, the shell would be wicked fast as it can run commands speculatively as you type them and discard the side effects if you backspace and memorize the results. Along with a zillion other game changer benefits like determinism. Built in perfect auditing. Rewind and restore processes to past states. That's what we should be striving for. But it's a lot harder.

leephillips5• 10 years ago

Curious if anyone here has adopted the shell mode of iPython as their routine shell (or something similar). Is it a good idea? Experiences?

gravypod3• 10 years ago

What is this intended on fixing?

I don't think the solution to anything is to make the shell more user-friendly. The only solution is to make the graphical user-interface more user friendly and feature full.

The only time it should be acceptable to be forced to use a shell, in my opinion, is if you are swapping out your desktop environment.

Until that happens, Windows and OS X will rule the desktop and PC market.

I'm fairly certain that Linux-based distributions will win out in the end if we can overcome this clingy gravitation to a command line. It's already happening already. Android is already winning against all of the major phone platforms and it's not because of it's award winning terminal emulator.

regularfry2• 10 years ago

The only solution is to make the graphical user-interface more user friendly and feature full.

There's only one feature that matters here, really: composability. Provide an environment in which I can compose simple tools into complicated workflows I can fling data through, and we might have a winner.

The only time it should be acceptable to be forced to use a shell, in my opinion, is if you are swapping out your desktop environment.

Why should that be true? It's not at all obvious to me. I have yet to see a GUI that lets you express something like `grep '^127' /etc/hosts | awk '{print $2}'` in any sane fashion. The closest you get is something like LabView, but that's counted out here because it's a general purpose(ish) language.

If your response is that this type of task is artificially skewed towards a text-based, shell-centred environment, then you don't have the type of problem that this shell is intended to fix.

gravypod3• 10 years ago

I have yet to see a GUI that lets you express something like `grep '^127' /etc/hosts | awk '{print $2}'` in any sane fashion.

I think we are in a very interesting time in computer science history where he have a huge slew of technology that solves a large set of problems that no one wants to exploit.

I could easily see an "assistant" type window being created in a desktop UI. Since we don't need a "windows" key in the linux world, I'd assume that would be used.

My ideal system would allow me to go into a text document, click the windows key, and say "grab me every line from /etc/hosts that starts with 127"

When the job would be completed, I'd see a check mark or something and the result would either be pasted to my cursor or, if my cursor wasn't in a text box, copied into my clipboard or something.

While this might sound like magical science fiction, it isn't. Neural networks, NLP, and HCI fields are doing some amazing things. The only problem is these are harder to implement then a simple terminal and pipe.

That is a sane way of interacting with a system that is user friendly.

In the same way, it could also handle other complex things. IE: "open my browser", "open the Internet", "list all the things connecting to the Internet", "open the last document I was editing in my text editor"

This is doable, albeit extremely difficult to implement well, but doable.

JdeBP4• 10 years ago

... although that is a canonical Useless Use of grep. (-:

* http://porkmail.org/era/unix/award.html#grep

maho2• 10 years ago

My number-one wish for a NGS:

Undo!

Take, for example, rm. The hoops we have to jump through when accidentally rm'ing a file are ridiculous [1]. But in most cases (smallish, non-secret files), rm should be trivially undoable. Windows gets this right: By default, files are not deleted, but moved to trash. If there is not enough space in trash, Windows warns you. Or, if you really want to delete a file instead of moving it to trash, you can press SHIFT+DELETE, in which case Windows will also warn you that it can't be undone. (What is missing in Windows is a "nuke" option that overwrites the old bits with random data, for those rare cases where a file must be purged from the system completely.) But in most cases, after deleting a file, you can simply press CTRL+Z to get it back.

It is possible to make rm behave like that in Linux [2], but in a NGS, this should be the default behaviour (in my view), with "delete" for really deleting (unlinking) a file, and "nuke" for completely distroying a file, as seperate commands.

Undo is hard. Most programs are on their third, fourth, or even higher release before getting it right. (Mathematica 10 is stil trying to get there...) But I think we should try harder to solve the undo-problem with respect to file-system interactions, or with respect to system-settings, in Linux.

[1] http://unix.stackexchange.com/questions/10883/where-do-files...

[2] http://unix.stackexchange.com/questions/42757/make-rm-move-t...

CaptSpify4• 10 years ago

Although I kind of agree with you, I'll play the devil's-advocate: If you actually care about the file, it should have a backup that you can restore from anyway. And, depending on the file, that backup should be source-control.

blakeyrat2• 10 years ago

If the solution to the problem relies on human beings being fully-informed, entirely competent, and never making mistakes-- it's not a very good solution, IMO.

Yes, people should maintain backups. But if people don't, for whatever reason (maybe they've never heard the word "backup" before; maybe they typed the wrong command and thought they had backups all along, etc.) shouldn't be screwed if something goes wrong.

gherkin0• 10 years ago

I know you're playing devil's advocate, but that attitude is making the perfect the enemy of the good. Backups are non-trivial enough that many people don't set them up correctly if at all; and even if they do, a restore can take some work. I think it's perfectly reasonable to have relatively painless alternative (e.g. "undo_rm") as an extra layer of safety.

ansible• 10 years ago

Backups are good. More frequent backups are even better.

I have btrfs snapshots run on the fileservers at 10 minute intervals. Which, of course, captures the entire filesystem state, not just files that you have deleted by accident.

koolba• 10 years ago

I bet you could solve this with ZFS and automatic snapshot creation for each new command. Want to undo the past N commands? Roll back to a previous snapshot!

i336_• 10 years ago

And a policy mechanism that lets you define how important a directory (or file) is and how hard the system will work (eg removing less-important data to free up space) to save this.

This idea falls over if the amount of data you create (which the shell can't know in advance) is too big to contain in a snapshot. All that would be possible is an automatic notification that the operations in XYZ directory are no longer undoable due to its size.

athenot• 10 years ago

On the flip side, you can have multiple filesystem entries (via hard links) to the same set of files and set up your own playground directory while keeping a mirror view of the same directory. Remove a file from dir A, and it's still in dir B. Once you're satisfied with the result, you can delete the entire dir B. Accidentally removed a file from A? Restore it from B by making another hard link. (Caveat: hard links don't work with directories unless you're Apple TimeMachine)

Natanael_L• 10 years ago

It would require the programs to explicitly describe their actions to the shell and by default behaving in a reversible manner. That would allow you to single out effects you dislike, and tell it to undo it.

Kind of like making the shell behave in relation to your OS as an editor/IDE + version control system, rather than just as an editor/IDE. Old versions are preserved until you purge them.

Or perhaps it would just simulate state changing actions by default, waiting to be told to execute them.

Arnavion• 10 years ago

Windows gets this right: By default, files are not deleted, but moved to trash.

The del or Remove-Item commands also permanently delete the file, just like rm. The "Windows" behavior you're talking about is the behavior of the graphical shell Explorer, which is also present in Gnome and KDE. There's nothing specific to Windows here.

shawnz• 10 years ago

That said, i'd like a "del to recycle bin" in Windows too.

Symbiote3• 10 years ago

  apt install trash-cli
  alias rm=trash

Done? This implements the Freedesktop specification, so is compatible with what KDE and Gnome do.

https://github.com/andreafrancia/trash-cli

maho2• 10 years ago

You are right, I forgot about Gnome and KDE. I tend to use classic shells on Linux, and the graphical shell on Windows, so I made a wrong generalization. It's probably more accurate to say that "undo" of simple operations (rm, mv) is something graphical shells get right.

But why are classic shells shipped without even the simplest "undo" features? We can probably all tell stories-from-the-trenches of how we accidentally did something aweful by typing the wrong command. Sometimes, it's hosing the network configuration of a remote machine, which is not something where "undo" would help you. However, in my experience, most cases involve doing something foolish with rm or mv, where "undo" would be tremendously helpful. I guess the main reason why I don't use the text shells of Windows is that Windows explorer gives me Ctrl+Z, which is saving my behind approximately once per year.

As a final thought, all shells (graphical or not) should expand the undo-features beyond rm and mv. For example, if I change system settings (or even Application settings?) and am not happy, I would love to be able to simply "undo" them, without trying to remember what the previous setting was. I realize that this is an incredibly hard problem, and that it is unlikely to be solved in an evolutionary step of the existing shells. That's why hearing about NGS got me so excited.

fao_2• 10 years ago

If I haven't misread, you're proposing a replacement to rm that has support for undo? Why not just make a shell script that uses mv to place the item in, say, ~/.deleted and drop it in your PATH?

ParrotyError• 10 years ago

Back in the 90s (IIRC) someone wrote a kernel module for linux that provided the equivalent of a GUI "trash can/recycling bin" but at the filesystem level, where a specific directory (folder for you young 'uns) was set aside for the deleted files. Because it was a kernel module (filesystem driver) it was transparent to applications, including the shell.

botw• 10 years ago

lost+found? it's still there in each file system but i never see a real usage for it.

Gracana• 10 years ago

The first few bullet points (regarding job control / status / blocking) sound interesting and useful for anyone. I think it would be cool to have a shell with some sort of visual job control system and an easy way to background anything that was taking significant time. Maybe that wouldn't be as deeply integrated as the system the author is describing, but it would probably be easier to implement and less of a change from a standard shell. These days I hardly ever suspend a process and continue it in the background because it's easier to spawn a new shell, but if that workflow were streamlined... maybe I wouldn't need a dozen shells open all the time.

As for the rest, I dunno. Neat, but complex solutions like this make me pause. Maybe instead of trying to solve this problem, we should instead try to not have this problem. That is, if your tools require this much of your shell, maybe the tools are the problem and not the shell?

isaaaaah• 10 years ago

My 2 tinfoil hats: If they can't have your kernel, they try to get your shell...

To me, i really do not see the points listed to be missing. For instance my shell script for remote execution on multiple hosts needs somewhere around ten lines of code and a peer list. I would never outsource this to an external entity. Especially today.

htor• 10 years ago

The irony...

Next generation UNIX shell. See the man page.

kps4• 10 years ago

Actually rc was the next generation Unix shell.

(Personally, I use ksh, which, while as bloated as bash, is bloated in more useful ways, and not so gratuitously divergent from plain sh.)

skywhopper• 10 years ago

There are lots of mixed-up goals here that I think might be better served broken out into multiple projects:

1) Terminal UI improvements 2) Interactive shell UI improvements 3) Shell scripting language improvements 4) Userspace utility improvements

Tackling all of these together may well be possible, but it will likely limit the potential impact of any improvements you are able to make. IMO, it's easier to move the status quo with incremental improvements, not wholesale re-imaginings of how the entire text-based Unix user interface works.

Ultimately, the most dangerous thing to mess with is the language itself. The strongly-typed Python-lite described in the linked man pages is not going to replace sh/bash for simple scripts, sorry. For most shell-scripting needs that sort of thing is wayy overkill. Sure Bash is ugly, but ultimately there's no way to harmonize quick-and-dirty command-line compatibility with an elegant scripting language syntax.

An alternative approach here would just be to expand your thought process a bit and don't get too worked up about Bash itself. Pipelines and new tools can solve all your problems if you're willing to think about them in a different way. `jq` is an amazing pipe-friendly JSON munger that is now just as essential to my CLI toolbox as `awk` and `sed` ever were. It solves your structured-data and typing complaints without interfering with Bash one way or another.

ilyash9• 10 years ago

The strongly-typed Python-lite described in the linked man pages

It's not strongly typed and types annotations can be left out in many places:

mylist.map(F(elt) "my $elt")

elt type is not specified.

The command syntax $(...) will cover some of the simple scripting needs. Please provide specific code examples (probably in bash) that you think will look bad.

`jq` is an amazing

It is but in my opinion much less convenient than built-in support for data structures.

vox_mollis2• 10 years ago

[WARNING, RANTY]

This is a blue-sky project. Completely new, with no legacy dependencies.

AND YET THE DEVELOPER CHOSE TO WRITE IT IN C. WHY FOR THE LOVE OF GOD, WHY?

Could someone explain this to me? Is it developer hubris, believing in one's own infallibility that a single exploitable stack frame or buffer overflow couldn't possibly happen "on my watch"?

If you need a native binary, your options are endless. Rust, Go, Haskell, hell, even C++ with smart pointers and runtime bounds-checking would be a step up.

Please, someone educate me on why people still choose to write inevitably-vulnerable software in 2016, when there is no legacy reason to do so.

leephillips5• 10 years ago

Do you think it's a bad idea that git and the Linux kernel are written in C? Is the possibility of a buffer overflow important in a program that's not processing input from random people on the internet?

codygman2• 10 years ago

I wonder if git would be better served by a more modern language.

vox_mollis2• 10 years ago

No, because they are both legacy software. I understand momentum in codebases - that's why I'm reserving my vitriol for fresh bluesky projects like this one.

As for exploitability, privilege escalation and shellcode injection is still very much a thing, internet-facing or not.

typon• 10 years ago

I'm sure you've heard of the Shellshock bug.

ilyash9• 10 years ago

See the full answer here: https://github.com/ilyash/ngs/issues/3

ilyash9• 10 years ago

Thanks for your input and GitHub stars! You are all very welcome to open GitHub issues with your suggestions.

I'm looking for developers to join the project. There is a lot of work to do :)

Ilya Sher, the author.

rhabarba• 10 years ago

Because lol, standards.

nikolay• 10 years ago

There've been many attempts and no web-based shell is the next generation - only the next toy. A replacement of Bash, Zsh, and Fish is needed, it just won't be a web-based toy!

nickpsecurity• 10 years ago

Use a general-purpose language with macro's or DSL's. Racket and Rebol are older ones with RED language being designed partly to do what article describes.

https://en.wikipedia.org/wiki/Red_%28programming_language%29

chickenbane• 10 years ago

If you're into Scala, Haoyi Li has a great video about his (similary) Ammonite project

https://www.youtube.com/watch?v=dP5tkmWAhjg

http://www.lihaoyi.com/Ammonite/

fao_2• 10 years ago

I think an OOPish DSL as a shell is pretty wrong[0], to be honest. Most of the time in UNIX what you're doing is chaining commands or automating. So perhaps what's needed is a DSL with a more functional, stream-based approach. Something along the lines of:

map `rm -rf ` (grep "*.tex" (ls))

[0]: I haven't had much experience with FOOP, however.

NGS: Next Generation Unix Shell

Related Stories