A year of building for the terminal

https://textual.textualize.io/blog/2022/12/20/a-year-of-building-for-the-terminal/

by willm • 4 years ago

278 88 4 years ago

textual.textualize.io

Wow, I used to like midnight commander a lot but moved away from it. Seeing things like that being built relatively easily makes me want to try using this for my personal project! Good work.

kredens• 4 years ago

cool! something like this for golang?

eliaspro• 4 years ago

Check out the Charm libs by https://charm.sh/

kjcondon• 4 years ago

This is an interesting framework, it's refreshing in that you don't need to bootstrap a whole web browser (electron) to build a user interface.

I don't really see it's use case though, mostly cli is used for either:

A) Ability to run in automated environments (e.g CI) - think aws cli

B) Interop, think piping data from grep to xargs

C) Hacking - writing a script quick and dirty to just get a job done

I'd have thought if I was investing enough time to think about user interface, I'd just build my tool in Qt/GTK/JavaFX/Electron etc

Though I can see the argument that if I was building an app with keybinds as controls rather than mouse primarly, probably a framework like this would be the way to go.

almostjson• 4 years ago

The uses for the terminal outlined are pretty limited. These are kind of the use cases I would expect from an average use that runs scripts or does occasional scripting work. It doesn’t really take into account more advanced users who do practically live in the terminal and do all the things outlined in the article (use a file browser from within a terminal)

The target users here are pretty clearly them. It’s pretty hard to make usable tuis because the libraries kinda suck or are ancient - this project seems to be looking to change that.

I’d even argue that the reason more people don’t use TUIs as much as GUIs is because TUIs aren’t always as intuitive or user friendly. Hopefully this helps that and makes them easier to build usable intuitive guis

vageli• 4 years ago

Have you ever heard of the curses library? There are use cases where you want to provide a graphical interface of sorts to a cli program.

https://en.wikipedia.org/wiki/Curses_(programming_library)

__MatrixMan__2• 4 years ago

I spend a lot of time with k9s, a TUI, and if there's a browser based or native GUI version, I don't care.

Terminals provide a uniform experience everywhere, more or less, and they compose better than browser apps. I can run tmux in asciinema and from there I can ssh elsewhere and use k9s. fzf and my zsh history make it so that it requires a few keystrokes to do anything (contrast this with being at the mercy of several conflicting designer opinions). A bunch of browser tabs is a comparatively clunky experience.

If web or native GUI apps interoperate, it's because their authors had a conversation. Terminal apps interoperate by accident.

jrmg• 4 years ago

Though I can see the argument that if I was building an app with keybinds as controls rather than mouse primarly, probably a framework like this would be the way to go.

Those old terminal apps that used to be used in banks, car hire places, airports, pharmacies and the like were _extremely efficient_ to use when you got to know them. Like, anything could be done with a few muscle-memory keystrokes. I always think of them when I’m in a line and an employee is busy peering and mousing around.

I’m not sure how you could convince a company to go back to something like that though - it just _seems_ so unintuitively unapproachable at first. Maybe if you got a few high profile clients and they could publish how much more efficient they were (presuming they were)?

leejoramo• 4 years ago

I am someone who used micro-computers such as TRS-80, Apple ][, CP/M and early MS-DOS PC's. I used VisiCalc, MultiPlan, the original MS Word, dBase and many more. One of the most significant pain points with such apps was the lack of stand UI and keyboard conventions.

One of the revolutions of the Macintosh was not just the GUI and Mouse, but the standard visual UI and keyboard shortcuts.

Borland's Turbo Vision almost got us these features for TUI, but it was too late. I am glad to see people taking another crack at it.

So let's make this TUI consistent with the conventions of your preferred GUI. I personally prefer macOS, but I am currently logged into to Windows 10 and KDE Neon. I spend most of my day in VSCode, with lets me configure it any way I want to.

troyvit• 4 years ago

I worked for a company that needed a bunch of tools developed quickly for internal support and implementation teams. We didn't have the authority or the time to build many web apps but it was super easy to set up a git repo with a bunch of cli apps for the teams to use. Everybody picked it up pretty quickly and felt pretty cool using the terminal to accomplish awesome things.

modernerd• 4 years ago

Has TUI accessibility improved at all?

Textualize (Python), Charm (Go) and tui-rs/Cursive (Rust) make building TUI apps appealing, but I'm put off by the idea of making apps like this if they have no screenreader support.

sekao2• 4 years ago

I think we will need new standards here, because terminal emulators currently don't have any semantic information about what TUI programs are rendering. Browsers have the benefit of HTML to know this is a button, and that is a paragraph. A terminal is an unstructured grid of characters. The best we can hope for is for terminal emulators to use heuristics or AI to make sense of what TUI programs are rendering.

willm11• 4 years ago

Textual has a browser-like DOM internally, which means that it has the same kind of structured information that a screen-reader would need. But there isn't a screen-reader API for the terminal to expose that.

On the roadmap is a browser target for Textual (same API, just rendering in the browser). Once that's available Textual can make use of browser technology for accessibility.

sekao2• 4 years ago

The browser target sounds neat. Do you plan on running the python in web assembly, or would it be a client/server design?

samwillis4• 4 years ago

I'm intrigued what the planned architecture is for the web front end. I'm guessing WebSocket back to a persistent process on the backend/server?

So all operations happening via the websocket connection. Only the GUI in the browser? I bit like Phoenix LiveView.

This would be a really interesting toolkit for building real-time UIs for remote processes.

cbm-vic-20• 4 years ago

I once asked the dev relations person of a TUI framework company if their stuff worked on a real physical terminal, and they didn't even know what I meant by that. Back to the nursing home for me!

UweSchmidt• 4 years ago

The terminal still has and always will have great potential as the glue that holds all computing together. Users refine their skill in a few, interoperable and flexible standard tools, and any command that turns out to be useful can immediately be part of a script that automates and simplifies this task forever.

It all hinges not necessarily on more fanciness in the terminal, but on the availability of data to work on, on other systems to open their data to the terminal in some way. Many SAAS tools have no interest or incentive to do that; is the trend going away from data available the terminal? Example, Kibana and Splunk are silos that provide powerful ways to analyze logs - what else could you possible want more?

Well, even the most sophisticated logfile queries can be repetitive, and those advanced search patterns could be distilled into a script that follows the user's hunches, add a bit of an ad-hoc regression test, correlate it with data from elsewhere, take user input from log files and use it in automated tests, maybe feed it all into a terminal based statistical analysis tool? I don't see myself doing any of that, the devs want a clickable link to the relevant Kibana and they are getting that.

More colors in terminals are nice but is it enough to make a difference?

zokier• 4 years ago

The terminal still has and always will have great potential as the glue that holds all computing together. Users refine their skill in a few, interoperable and flexible standard tools, and any command that turns out to be useful can immediately be part of a script that automates and simplifies this task forever.

But the topic here is exactly the sort of framework whose apps are not particularly easily scriptable and integratable in typical shell-driven workflows.

pjmlp2• 4 years ago

Looks like I am reading an UNIX magazine article back in 1992.

sebosp• 4 years ago

I've been playing around rust generative art (nannou) and shader toys on top of alacritty to experiment with other types of visual feedback and other types of interactions in the terminal, not just at GPU level, granted early stages make it convoluted but IMHO still bearable, https://github.com/sebosp/chartacritty

ryanianian• 4 years ago

How does Textualize make money?

I played with Textual and Rich over the weekend. I found it super compelling to use TUI forms to generate json or other kinds of configs. I particularly like the faux CSS for styling and modeling UI updates as reactive components.

Before I invest further with it, I'm curious where this is all headed and if there will continue to be sustainable development in the future.

willm11• 4 years ago

We were funded with Venture Capital. Which means full-time developers for the foreseeable future. But its worth pointing out that both projects existed for a long time before funding.

thejosh• 4 years ago

And how do they see themselves getting money back? Is there a plan for paid subscriptions or something in the future?

Not trying to be snarky or anything -- generally curious! I love the project.

pipo234• 4 years ago

I think ultimately, the idea is some sort of freemium model where the TUI is free, but usage as a web UI is will cost extra(?)

willm11• 4 years ago

Kind of, yeah. But there will also be a generous free tier.

goffi• 4 years ago

Textual looks nice. I've been a happy Urwid user for years, can someone with experience with both can do a quick comparison? Urwid development seems stalled (no commit for 6 months). Is it now possible to embed a terminal as a widget with Textual (with Urwid it's possible)?

I'm wondering also how Rich compares to something like Python Prompt Toolkit.

enricozb• 4 years ago

It's not quite ready yet, but I'm writing something similar for Rust [0]. It's much more similar to React (with respect to components, and hooks), and the DSL is inspired by SwiftUI.

Disclaimer: Version 0.6.x has a pretty severe flaw where the constraints around hook order are way too strict, and this also prevents conditional renders [1]. I'm working on a full rewrite which will be released as version 0.7.0, which exists on github as a branch right now [2]. Docs also exist for an alpha version of 0.7.0 [3].

[0] https://docs.rs/intuitive/latest/intuitive/

[1] https://github.com/enricozb/intuitive/issues/4

[2] https://github.com/enricozb/intuitive/tree/0.7.0

[3] https://docs.rs/intuitive/0.7.0-alpha.0/intuitive/index.html

alganet• 4 years ago

This is great work

distcs5• 4 years ago

Impressive work! I am not trying to diminish the awesome work done here but I think all of this is already possible in Emacs, right? And Emacs an be made to run in Terminal if the user wants it to, right?

Now I am not someone who would ask creators not to build something new because something like it already exists. By all means build new stuff and create new tools. It providers options and choice for user. So I fully support work like this.

But I also ponder about user behavior. New users who flock to the modern terminal tools need to learn new keybindings, new shortcuts, new workflows. Yet they are willing to invest that time in learning these modern terminal tools. But many of the same users are averse to investing that time in Emacs. Why is that? What explains this?

EDIT: Downvoters, care to explain what about this comment violates this site's guidelines? My comment is an honest question. You can post comments and explain if you disagree with my premise. But downvoting isn't helping me learn anything.

rmorey• 4 years ago

I’m not sure emacs is a good comparison here - Emacs is a complete environment and set of capabilities, whereas this is a library/toolkit for Python, much smaller/ more specific scope. I think it may be possible to take a python script, and create a UI for it in emacs, but I think the goal here is an python-native solution to turn a python script into a TUI

distcs5• 4 years ago

Emacs is a complete environment and set of capabilities

But so is the shell and the terminal and yet new young folks are much more willing to try out shell+terminal tools than Emacs+packages.

Your comment describes the difference between this toolkit and Emacs perfectly from a dev POV and what you say makes complete sense. But I was pondering from user POV.

I mean when devs use this library/toolkit and create terminal apps, I am sure those terminal apps will find many enthusiastic users. Many of the same users would not find themselves so enthusiastic if Elisp devs provided the same tool to them as an Emacs capability. You see my point? I don't know why this difference in attitude towards terminal tools and Emacs tools exists. Is it because of the reputation of Emacs being old? Or its reputation of unusual keybindings?

ckolkey• 4 years ago

My pet theory is that its to do with the rate of new developers to the field. I think the accepted figure is something like "the field doubles every five years". To me, this explains things like idea-churn, wheel-reinvention, and "grandpas tools", where you see tried-and-true tools (emacs, in this case) passed by because they lack modern trappings, or because people just never hear about them.

davepdotorg5• 4 years ago

I mean, nice idea, but I'm a 55 year old Emacs user and I work on Textual.

tyingq4• 4 years ago

The blog post is from a team that builds a library for creating rich text mode applications in the terminal. The examples include a file manager, a diff tool, a floating gutter, dropdown autocomplete, animated underlines, pixel editor, and so forth.

Your comment is confusing because emacs isn't a library for creating rich text mode applications. While it has some things that are somewhat similar to things mentioned in the blog, it's an editor at heart...not a rich text/tty library for other tools to leverage.

throw10920• 4 years ago

emacs isn't a library for creating rich text mode applications

It (almost) literally is. You ever hear that old joke "Emacs is a great operating system, now if only it had a decent text editor"? That comes from that reality that Emacs is better at creating "rich text mode applications" (e.g. org-mode, SLIME, magit) than it is at actually being a good out-of-the-box text editor.

Emacs gives you a text-based abstraction that exists on a bunch of platforms, an extensive standard library, and a massive ecosystem for building text mode applications. Including, yes, "in the terminal".

MonkeyClub• 4 years ago

emacs isn't a library for creating rich text mode applications.

One could say with at least equal merit that Emacs is exactly that: a language and library for creating rich text mode applications, itself being the first application written in it.

To consider Emacs as merely an extensible text editor would be to miss out on what can be done with the digital text metaphor beyond its editing.

distcs5• 4 years ago

Your comment is confusing because emacs isn't a library for creating rich text mode applications.

But all the functions Emacs has and Elisp together do seem like a library for creating rich text mode applications. Just look at how Magit does text-mode and rich-text visualization of Git commits, trees. Look at how Org-mode formats headings, code blocks, tables in different styles.

pjmlp2• 4 years ago

They could have reached for Turbo Vision and call it a day.

civopsec• 4 years ago

Your comment is confusing because emacs isn't a library for creating rich text mode applications

The point isn’t what it is. The point is presumably that it can do this task better than the terminal.

johanvts• 4 years ago

I think all of this already possible in Emacs, right?

Dired, diff, Linum. Seems to cover these examples pretty well.

I think most people think of emacs as an overly complicated text editor and don't really ask themselves if they should consider learning emacs when they want a diff tool or a file navigator.

__MatrixMan__2• 4 years ago

I think the point of textual isn't so much "look what you can do with it." Most of what it does is possible with ncurses also, if you're really committed. But neither ncurses nor emacs feel like modern UI development tools, and that feel (e.g. styling with css) is important to a lot of people.

samwillis4• 4 years ago

EDIT: Downvoters, care to explain what about this comment violates this site's guidelines?

Downvotes on HN are not limited to "moderating", users are entitled to use them as they wish, many use them to mark that they disagree with a comment. I think people are disagreeing with your premise, or think you have misunderstood what Textualize is.

The guidelines do however suggest not commenting on voting on your comments:

Please don't comment about the voting on comments. It never does any good, and it makes boring reading.

https://news.ycombinator.com/newsguidelines.html

samwillis4• 4 years ago

But many of the same users are averse to investing that time in Emacs. Why is that? What explains this?

The issues is somewhat explained with how you form your question here, you frame it a little as "the users are wrong for not seeing that X is better than Y", that has a habit of pushing people away.

Ultimately is comes down to branding and communication, I have never learnt Emacs because it seems (from the communication around it) to be an elitist and complex ecosystem. I suspect I am not alone.

asah• 4 years ago

Actually, you got my upvote - emacs solved this problem 3 decades ago (!), runs everywhere, is easy to extend, and I'm gonna wager that emacs has far more users than these bespoke terminal programs, which come and go.

Countries will rise and fall, the sun will fade, and all that will remain are *nix and emacs.

dec0dedab0de• 4 years ago

I've been using terminals for over 30 years, been using linux and other unixes for over 20 years, and I have attempted to learn emacs multiple times. I have never been able to get to the point where i felt comfortable at all using it. In contrast with vi/vim I felt comfortable editing files within 10 minutes. So while I'm not a new user, I think I can offer some anecdata.

Attempting to learn emacs feels like wanting to watch spiderman no way home, and not knowing which of the hundreds of hours of related content you should watch first. Some of it makes your experience more enjoyable, Some of it is barely related, some of it is actively shunned by the creators, some is largely considered required viewing by the community, but not necessarily by the creators.

Emacs has decades of history, plugins and configurations that are entirely overwhelming to someone coming in new. There are a few tutorials out there, but all the ones I have tried felt strangely opinionated, and out of date. One of the first things I learned was that the keybindings are designed for a keyboard that no longer exists, but for some reason the documentation uses the terminology for that keyboard. Why must we read about meta keys when none of us have a meta key? Another thing I learned is that there are so many configurations and plugins, that there are distributions of emacs that come with modified defaults, but that you shouldn't use them until you take the time to learn why you need all of those plugins and settings. But who's advice should I take?

I think for emacs to gain market share there needs to be an official modern tutorial that holds your hand, and walks you through all the basics, and common configurations, including the top X most important plugins and how to use them. All while focusing on the actual benefits over the alternatives, without glossing over it's own negatives. It would need to be written by an emacs expert that is not an emacs zealot. I don't know if such a person exists, I have certainly never met one. It also would need to be updated atleast once a year to keep up to date with changes in the plugin ecosystem.

That is to say, it needs to feel like a thriving active project that wants new users. Instead it feels like a neighborhood pub with a sign that's falling down, and you have to go in the back door because the front one is always locked these days, but all the regulars know that so there is no point in doing anything about it.

margarina72• 4 years ago

Many options exist, but in case you are building in python, this is an awesome library. Charm in goland is what I can think of that come as close to it, and in elisp of course you would do that with emacs. While emacs is awesome, not everything that can be done in emacs would be an optimal solutions for all use case. Always use the tool for the job.

cryptonector• 4 years ago

Downvoters, care to explain what about this comment violates this site's guidelines?

Downvotes don't have to be only motivated by the downvoted's breaking site guidelines -- that's what flagging is for. Downvotes can be for any reason.

kleinsch• 4 years ago

You could invert everything in this comment and it would be just as valid. If you built this in Emacs, someone would comment: all this is possible in the terminal, new users who flock to Emacs won’t learn the terminal ecosystem, etc. Anything related to Emacs, Rust, JavaScriptjust turns into a religious “use this instead of that” opinion debate where nobody is wrong or right.

Use the tools you want to use. Don’t hassle other people for using different tools.

distcs5• 4 years ago

I don't see how inverting this comment would make it valid.

Users seem to like terminals already. The same users who like terminals still turn out to be averse to Emacs.

So if I invert the comment and ask, "Why do you new users who flock to Emacs won't learn the terminal ecosystem", the question would be nonsense because the fact is that new users do not flock to Emacs.

But new users do flock to terminal and still would not touch Emacs with a 10 foot pole. So my comment makes sense only in one direction.

JulianWasTaken• 4 years ago

Do you know of an example of a commonly used application which uses the emacs-driven functionality you're referencing, but where the application isn't generally used inside of emacs itself? I.e. do you know of a widely-used program which uses emacs as a library, but not as the top-level application?

This is a slightly leading question -- I certainly don't use any such applications, and am not really familiar with any, though I can't say I've ever looked.

But I think the above, as other commenters have mentioned, has a lot to do with why this isn't really a good comparison.

Even more so given that even if such applications do exist, programmers definitely like using libraries written or wrapped in their host language over ones simply available over FFI, so someone'd still want the functionality from elisp wrapped in a native API (a Python one in this case).

forgotpwd162• 4 years ago

I think the question they're making is why a shell-oriented ecosystem usually preferred over an Emacs one. Or differently why most users prefer an environment consisting of distinct libraries/apps rather Emacs and packages made for it.

robenkleene• 4 years ago

Emacs is great and flexible software, but maintaining an Emacs install for the features described in the blog post, if you're not already using Emacs as a text editor, is a bad idea in my opinion for these reasons:

1. Emacs (with a useful number of plugins installed) is slow to launch, fine if you're using it as a with `emacsclient` as a server, but using it for one-off use cases like these would be unacceptable due to the slow startup time. (E.g., `nnn` as a file manager in contrast opens instantly.)

2. Emacs does not compose well with other Unix tools if you aren't already running Emacs. E.g., you can't pipe to Emacs and process STDIN, this makes it a terrible fit for the diff example from the post.

3. Accomplishing these tasks from a terminal prompt is an unacceptable number of key strokes. E.g., you'd have to launch Emacs itself, and then run at least one other command to perform any of these functions. Simply unacceptable for those of use that value efficiency (e.g., produce as much output as possible per key stroke.)

Frankly, none of this should need to be said, but the highest voted comment on this thread doesn't acknowledge Emacs deficiencies for the use cases described in the blog post.

forrestthewoods• 4 years ago

Looks nice. But I’m not sure it’s worth the effort to build a “heavy” app in a TUI over a native app in Dear ImGui or similar.

I’ve used Araxis Merge as my diff tool for over a decade. It’s great. I don’t think there’s significant value in reimplementing something in a TUI just for the sake of it.

Nothing wrong with new tools to make building TUI’s easier. I just wish the examples were providing new capabilities rather than old capabilities in a very similar but slightly different form factor.

tux• 4 years ago

First time I heard about Rich and Textual. Very impressive work, will definantly use it for something. Thank you to all involved.

willm11• 4 years ago

Thanks. I will pass that on to the devs.

krunck• 4 years ago

"Please use animation sparingly."

So... Doom?

tambourine_man• 4 years ago

I’m really exited about the work you guys are doing.

That tab and easing animation is jaw dropping. I can’t shake the “that shouldn’t be possible in a terminal” feeling. In a good way.

And git diff | dunk seems really useful.

Keep going and posting.

willm11• 4 years ago

Thank you!

carapace3• 4 years ago

I went through a TUI phase recently and investigated a slew of these projects. In the end, I just used ncurses, there's no real point in using anything else. The biggest problem (that also affects Textualize) is poor or lacking documentation. But really the fundamental issue is that ncurses is all you need and it already covers all the edges and weird terminals etc.

davepdotorg5• 4 years ago

How long ago did you check the documentation? As of October this year the first full set of docs were published, with more to come: https://textual.textualize.io/

carapace3• 4 years ago

I just checked my browser history and my latest visit was October 1st. I must have just missed it.

jarbus• 4 years ago

I have a very specific use case: I want to have a UI interface around images generated by the kitty graphics protocol. Kitty, the terminal emulator, can render images in full resolution, which has been so helpful for viewing plots over ssh. I made a hacky ui for generating plots as I need them using prompt-toolkit, but having a proper TUI library that supports images would be so much nicer

davepdotorg5• 4 years ago

This sort of thing is on the development roadmap. Also, some people have been building their own version of this with Textual already. Few days ago someone was showing off a Textual app they’d built that would play video - using the kitty protocol IIRC.

blooalien• 4 years ago

@willm Textual (and Rich) freakin' rock! Thank you so much for your hard work makin' them great!

willm11• 4 years ago

De nada.

dlqx• 4 years ago

To be honest I am wondering what these people did not understand about terminals?

A terminal is not meant to resemble graphical UIs with all those whistles and bells! Looks like I am the only one who is wondering :D ;)

fragmede• 4 years ago

Ugh no. Just because choose to use a keyboard-based interface doesn't mean I don't want things like colors and graphics.

ibz2• 4 years ago

What is a terminal meant to do then and who says that?

Actually I would argue that graphical UIs copied terminal UIs that existed first and in some cases they did not catch up even as of now. See file managers. There is Total Commander on Windows, which managed to surpass the old Midnight Commander in functionality and ease of use, but that's about it. Very hard to find a decent file manager on Linux or OSX that comes even close to mc.

A year of building for the terminal

Related Stories