Fix Terminals

http://www.leonerd.org.uk/hacks/fixterms/

by smartmic • 5 years ago

135 65 5 years ago

leonerd.org.uk

tyingq4• 5 years ago

Typing "echo <ctrl-v><esc>c<return>" to fix a garbled Vt100 like terminal remains embedded in my brain from long ago. Despite simpler things like "stty sane" existing. I don't know if maybe I was an admin on some box with a wonky stty, or why that's in my brain.

haecceity2• 5 years ago

Is <ctrl-v><esc> same as \x1b? Why are they same?

tyingq4• 5 years ago

Yes, 0x1b is an ascii <esc>.

<ctrl-v> essentially means "insert the next key I press verbatim". So, like in Vim, <esc> would switch modes, unless you press <ctrl-v> first. Helpful if you're trying to insert a control character into a file, or echo a literal <ctrl-c> etc.

VMG• 5 years ago

I just type `reset` (maybe it's a zsh thing)

tyingq4• 5 years ago

I don't remember, though I worked on a lot of oddball stuff. 3B2, Pyramid OSX, DGUX, etc.

drran2• 5 years ago

Why not just `clear` or `reset` terminal?

tyingq4• 5 years ago

Wish I could remember why not :). I know now, but the muscle memory sticks.

dan-robertson4• 5 years ago

Personally I feel like the fix for terminals is not to use them. Emacs for example has a lot of effort put into supporting various different terminals and efficiently displaying text. Some xterm-specific support is for many colours, mouse support, and the control codes to ask xterm to send more precise key escape sequences so TAB and C-i aren’t confused for example. It also has a gui which doesn’t need to pile on layers of hacks and works great so long as you don’t need to go through a terminal (eg running over ssh.) And yet it is still straddled with backwards compatibility from those days (some users expect TAB and C-i to do the same thing.)

To me it feels silly to put a lot of effort into supporting things better in terminal emulators because they are not as flexible as actual user interfaces and I think the main reasons we still have them are historical.

Aside: I’d also like to see a command line shell which runs outside the terminal emulator rather than inside it.

nixpulvis• 5 years ago

All the issues are just matters of backwards compatibility at this point. As far as I'm concerned, we'll always need a text based REPL oriented interface.

coliveira• 5 years ago

You can also learn to use the terminal. This can clear many of the problems that you would otherwise have by thinking that the terminal is just another windows/macos application.

dan-robertson4• 5 years ago

I don’t know how to respond to this as you seem to just think I’m an idiot who doesn’t know how to use computers.

The problem with using a terminal is that control of it is distributed between a few things the user can control and the escape sequences (or just output) produced by the shell and any processes that run in it. These may end up conflicting with each other leaving your terminal in a bad state or you may just get broken output (ever tried piping pv something | ... | less?). One way to regain some control is with something like tmux but this can become unwieldy (and heaven forfend it sees a multibyte Unicode character—terminal emulators don’t really have a way to communicate with applications about how wide a character is going to be when drawn)

I think a few things get conflated because there are few text-centric or command-line user interfaces. I put it to you that it is possible to have good composable text-centric user interfaces that don’t rely on pretending to be a VT100 or an ancient single-byte-stream-with-control-sequences protocol.

theamk• 5 years ago

The problem with GUIs (I assume you mean them by "actual user interfaces") is that they rely too much on the user's memory, and that they do not scale, especially for the cloud systems.

I have many text notes which record various commands. I have them because it is so easy -- once you figured out how to do something, get the history and copy relevant stuff. At work, we share the commands on Slack, put them into Wiki, paste them in the docs, and so on. If the commands grow too complex, they turn into scripts -- after all, scripts are just file rename away.

I don't think I could do this efficiently I had to use GUIs. Theoretically, I could write an manuals with dozens of steps[0], but this is much more significant effort, and they'll likely go stale anyway.

[0] https://docs.microsoft.com/en-us/iis/application-frameworks/...

zokier• 5 years ago

You are confusing command line and terminal here, I believe. Note how OP was specifically using emacs as an example. From your "store and share commands" point of view so called TUIs (like emacs in terminal) are as opaque as GUIs (emacs in X11).

simiones• 5 years ago

Note that nothing stops a GUI program from also supporting programmability and scripting. While it's true that most of them don't, you'll also find that a surprising number do.

For example, the entire MS Office Suite is programmable - you can write VB Script (or lately, JS?) and achieve most if not all of the functionality supported in the GUI.

Rather more well known on this front, Emacs and many other Lisp systems have always supported the same kind of programmability as a shell from within the GUI environment - both in the form of a simple REPL and more advanced GUI-command interaction (e.g. executing the current selection as elisp code, executing a command with the current selection as input etc).

muststopmyths2• 5 years ago

Aside: I’d also like to see a command line shell which runs outside the terminal emulator rather than inside it.

The problem with such a shell would be that all the simple utilities and tools one likes to use in a shell will not work without a tty (or pty or fake-pty-conhost.exe). So the shell by itself would be pretty useless.

This theoretical shell and its family of utilities would all have to emulate terminal emulators by using something of a common library of wrappers around tty functionality. At that point you've re-invented cygwin for your OS of choice. And you're going to be running the same kinds of stuff you'd run in a real terminal emulator.

I'm not being snarky. I gave this kind of thing a lot of thought many years ago when I was thinking of native GUI unix-style shells on Windows.

Unless you have a different use case in mind for this shell ?

dan-robertson4• 5 years ago

I’m suggesting running outside the terminal by which I mean pulling up new terminal emulators as necessary rather than running inside a single terminal emulator and taking over the whole thing for each command (and allowing applications to get it into a bad state). If you want to run less on the output of one command or watch some thing change while still running other commands you either need to open a new second terminal emulator and go to the same place as the first or you need to use something like tmux (and either have some janky hack or be very careful about when you run tmux if you want to duplicate shells on a remote box for example).

I also think it’s not so useful anymore to just set up a pipeline and let it run off and do it’s work. Interactive shell usage is often mostly about editing or extending a small suffix of the previous command, so I think a shell should be optimised to do that well.

jonjacky• 5 years ago

>Aside: I’d also like to see a command line shell which runs outside the terminal emulator rather than inside it.

Plan 9 did something like that with its rio windows (that replace the terminal emulators) and its rc shell.

https://9p.io/wiki/plan9/using_rio/index.html

http://man.cat-v.org/plan_9/1/rio

The problem with such a shell would be that all the simple utilities and tools one likes to use in a shell will not work without a tty (or pty or fake-pty-conhost.exe). So the shell by itself would be pretty useless.

Right -- no vi, no emacs, no readline, no curses -- nothing that uses cursor addressing. You have to start all over. Plan 9 can be seen as an experiment to find out how much you can simplify if you can abandon backwards compatibility.

opk2• 5 years ago

Aside: I’d also like to see a command line shell which runs outside the terminal emulator rather than inside it.

So how would that work over something like ssh or even a serial terminal. And you'd have a shell containing a lot of very platform specific code. That really breaks everything. I'd prefer to see well thought out evolutions to the ANSI escape sequence standards that solve real problems. The proposal on the original post does that for keys. Other things like bracketed paste and support for more than a few colours have seen some adoption. Some good ideas get less attention like having a stack for titlebar changes. Some things that weren't a great idea security-wise have been dropped (key redefinitions, retrieving the title).

One of the problems is that the concept behind termcap and terminfo doesn't really scale or allow for innovations. As a user of rxvt-unicode I often suffer the frustration of it being unrecognised on bare OS installs. But I respect the fact that it doesn't just claim to be xterm while emulating it imperfectly like many.

dan-robertson4• 5 years ago

Well no one uses a serial terminal for much so that can be disregarded. For ssh, you can just stop pretending that it is just like any other program and have actual integration with it (if you still care about serial terminals, just make the mechanism that understands ssh generic). On the remote host you either need your shell executable to run in some “remote” mode like rsync does or you need your shell to be able to generate appropriate bash commands.

Take eshell for an example: it connects processes together in emacs and can natively support navigating to places on remote hosts, opening files there, or running (remote) shell commands.

I sympathise with you on the pain of terminfo+ssh where the remote host doesn’t have the appropriate files. But I don’t think that further piling on “standards” to terminal escape sequences is a long-term solution to terminal woes.

vemv• 5 years ago

While I don't enjoy being negative, this doesn't sound like a solution that would work - it essentially needs 'boiling the ocean' i.e. convincing everyone else.

(That's literally the plan described in the article)

The first paragraphs of https://clojure.org/community/etiquette nail it: the software community is about making. If you want something done, make it.

How would a "maker-driven" solution look like in this domain? For example:

- make a terminal emulator that accepts the devised keybinding system

- make at least one popular program (e.g. Emacs) play fine with said terminal

- promote that combination, showing the world how the technology can in fact work. Word of mouth would do the rest.

aurelian15• 5 years ago

You are certainly not wrong. In this specific case however, I'd like to point out that the author, leonerd, has been busily working on "making stuff" to this end for years now.

They are the lead author of libvterm, a popular modular terminal emulator library that is for example used in neovim and emacs-libvterm.

They have also been working on libtermkey, a library that accepts input from the divised keybinding system, now part of libtickit mentioned in the post.

Scarbutt• 5 years ago

like this? https://iterm2.com/documentation-csiu.html

mixmastamyk2• 5 years ago

Any other terminals support it?

enriquto4• 5 years ago

seems cool but it is macOS-only

hultner• 5 years ago

Cool, so heat vemv requested already exists with iTerm and neovim.

gumby• 5 years ago

Excellent satire!

ashton314• 5 years ago

The conflation of various control characters, i.e. `Ctrl-i` and `Ctrl-I`, was one of the primary reasons that convinced me to try using Emacs outside of the terminal. I'm glad I switched. Now I can make much more ergonomic keybindings.

aidenn02• 5 years ago

I agree that using the 8-bit CSI to eliminate the escape ambiguity would be great. The rest I'm a bit "meh" on. In particular I use C-S as the modifier for my terminal shortcuts, so if the terminal could detect it I'd be looking for yet another set of bindings that no terminal programs will ever use.

alpaca1283• 5 years ago

The only decent solution in my view would be a new, separate kind of terminal spec, maybe even completely focused at terminal emulators. It could make sense to handle input better(maybe even with keyup/keydown events) and maybe display images in a non-hacky way(it can already be done via w3m...kind of).

Changing the existing standards just won't work as it would break a ton of programs, although if I'm being honest - while I like the idea of such a terminal rework - I don't know if it can be done nowadays without it opening the flood gates by developers who just can't restrain themselves from adding so many features it might as well be a new browser engine.

And either way I don't think it's worth the effort as those input limitations only affect a few specific situations. I use hundreds of custom keybindings in Vim and so far the only obstacle I encountered was the `Tab` & `Ctrl-i` overlap. If I need to hold down more than one modifier I did something wrong anyway.

aumerle6• 5 years ago

https://sw.kovidgoyal.net/kitty/keyboard-protocol.html

https://sw.kovidgoyal.net/kitty/graphics-protocol.html

mixmastamyk2• 5 years ago

I didn't fully understand the spec, it got hard to read towards the end. Is this is supported by curses/terminfo?

Animats• 5 years ago

You can work directly at the keycode level, and watch the key up/down events. Many programs do, especially games.

duskwuff• 5 years ago

That doesn't work in a terminal. Terminals can operate over many different channels which don't pass through raw keyboard events (like SSH), or which don't have them at all (like serial links).

alanbernstein4• 5 years ago

Can anyone point me to an article that explains why we are still stuck with these standards from the 1970s? I understand backward compatibility can be important, but I don't quite understand why terminal standards are the ONE thing in the computing world that just can not move into the future.

opk2• 5 years ago

Just be glad that it didn't go the other way with terminals being full HTML 5 and having to watch advert videos between command invocations.

m1gu3l• 5 years ago

i think a few years ago someone tried to get me to try some terminal app that was Electron based. not AS bad but still... barf.

enriquto4• 5 years ago

just can not move into the future.

It can. How would you like them to move into the future? It's not as if there's been no recent evolution in terminal emulators.

Notice that evolving into the future and using old ideas are not incompatible. Modern cryptography still uses Euclid's algorithm from more than two millennia ago.

AnIdiotOnTheNet• 5 years ago

How would you like them to move into the future?

For starters: stop using in-band signaling.

alanbernstein4• 5 years ago

Here's one example: Why can't I interact with text at the prompt in the way I do in every other text input context on my modern computer? Selecting text with shift and arrow keys, moving the cursor with the mouse.

Keep in mind that I do not deeply understand the stack of standards and implementations that comprise a terminal. That said, my assumption is that this complaint is an unfortunate, unavoidable byproduct of that stack.

While introducing newcomers to terminal usage, there are simple actions from the gui world that do not work at all, or even seem to break the terminal display. It's extremely confusing for them, they get over it eventually, and come to accept that the terminal just can't do certain things. This is what I mean by being unable to move into the future.

As a point of comparison, consider how much of a mess it is to deal with spaces in file names, in bash. To a newcomer, this seems like a crazy hassle, some antiquated nonsense. It's hard for them to understand how experienced Linux users can stand to deal with it. It's annoying, but it's a direct consequence of the choice to use the space character as the delimiter between tokens in bash. This choice is actually super convenient most of the time, because that is the best use of the space character/key in a terse shell language.

What is the analogous rationale behind the shortcomings of the terminal paradigm? What would we need to give up, in order to make the terminal interface a little bit more modern?

otabdeveloper4• 5 years ago

It works okay, unlike most of the rest of the crap in computing. Leave it alone.

alanbernstein4• 5 years ago

You're right! I just want to understand why it's stuck at ok, and has what seems like 50-year-old cruft that can't be touched.

micouay• 5 years ago

I'd also really like to just be able to interact with a keyboard like other apps i.e. receive keyup events, reading multiple keys at once etc. Kitty has something like that but popular libraries don't support it. I've had many ideas for terminals games and apps but didn't continue because of these limitations.

aumerle6• 5 years ago

This proposal is incomplete and has various bugs: https://sw.kovidgoyal.net/kitty/keyboard-protocol.html#fixte...

drran2• 5 years ago

I have few mobile devices with limited keyboard: smartphones with keyboard, chromebook. They lack some keys presented on a standard PC keyboard. Will this new input library work with virtual keyboard, limited keyboard? How I will be able to input a missed key or control combination?

qwerty4561272• 5 years ago

Indeed, terminals have to be fixed. Even DOS/Windows text mode (I don't mean the shell) always felt way better than a Linux terminal.

Perhaps it finally is time to come up with an entirely new terminal technology re-engineered from [almost] scratch based on today achievements and needs.

For a period of time near 2000 people felt like terminals were going to be abandoned in favor of GUIs so changing them didn't feel reasonable. But now it's clear that was wrong.

enriquto4• 5 years ago

Perhaps it finally is time to come up with an entirely new terminal technology re-engineered from [almost] scratch based on today achievements and needs.

What do you mean, exactly, by that? Do you think that programs designed to work in current terminals (e.g., vim) should stop working with the new terminal technology? That seems a bit tough for progressive adoption. Otherwise, how do you think that these programs should be changed? Or wrapped in some intermediary "terminal emulator" so that they can run unchanged in the newer terminals?

qwerty4561272• 5 years ago

Why not?

Also, the new terminal technology doesn't necessarily have to be fundamentally incompatible with the legacy technology. It probably can be made reasonably easy to compile classic apps for any terminal technology. Many of them are compatible (although not necessarily 100% compatible) with both Linux and DOS/Windows already.

bloak• 5 years ago

How does this interact with alternative and personalised keyboard layouts and with sticky keys for accessibility?

I don't have a proper overview of how these things work, but I'm a bit worried that there may be multiple layers (reprogrammable keyboard, kernel, X server, terminal, editor) trying to solve similar problems in similar ways and they might end up interfering with each other.

Fix Terminals

Related Stories