Why Is It So Hard to Detect Keyup Event on Linux?

https://blog.robertelder.org/detect-keyup-event-linux-terminal/

by EntICOnc • 5 years ago

156 128 5 years ago

blog.robertelder.org

akmittal• 5 years ago

All the applications getting keyup events seems like big security issue.

Is this same case with Wayland

yw3410• 5 years ago

Is what the same as Wayland? Maybe you've misunderstood the article?

The process is running as root and reading directly from /proc; the article acknowledges that it's essentially a keylogger.

chaorace• 5 years ago

I am now curious, though. How does Wayland handle key events? I have to imagine it only lets the active window listen in on key events, right? Maybe there's an API for subsribing to specific key combinations?

pjc502• 5 years ago

A tty doesn't know about the concept of key press or key release events. It only knows about 'data stream in' and 'data stream out'.

This is basically the long and the short of it; it's a very old API, and retrofitting key-up detection reliably is a serious problem.

CoastalCoder• 5 years ago

Perhaps this is nitpicking, but I don't think the issue here is "data stream" vs. not-"data stream".

Perhaps a better distinction is "data stream of characters" vs. "data stream that can express key events (and perhaps other stuff as well)".

swiley• 5 years ago

This is correct, plan9 continues using a data stream but puts events in the data stream rather than characters.

Even on Linux the low level kernel API for getting keypress events works this way.

icedchai3• 5 years ago

It's really a lower level protocol issue. The serial terminals upon which the "tty" model is based had no event stream outside of characters being sent or received. There are events provided by the serial port itself and delivered out-of-band though (example: modem DCD line drops, SIGHUP is generated.)

drran• 5 years ago

To fix this low level problem, someone will need to modify xterm, or libvte, or any other terminal, to send escape sequences for key release events, when such mode is requested by an application.

patrec2• 5 years ago

Already done. Kitty and other modern terminals have an extended keyboard interface which allows to detect keypress, keyrelase and arbitrary key combinations (unlike legacy vt100-style terminal emulators, where e.g. Tab and Ctrl-I are indistinguishable). For backwards compatibility you need to switch to this mode by sending an escape sequence.

https://sw.kovidgoyal.net/kitty/keyboard-protocol.html

ajross4• 5 years ago

tl;dr: It's not about Linux, it's the terminal.

Linux exposes your keyboards as /dev/inputN devices and (with the exception of oddball devices or drivers without the hardware capability to detect them) every single one of them presents clean down/up pairs on every press, every time. If you want to read the device state, read it from the kernel.

The problem is that the author doesn't want to detect the device state. The author wants to write a program to run at the command line. And the command line is an environment descended from a long line of tty environments going back to the original teletypes of the 1960's. And those devices were never designed to expose a "keyboard" as a "device". They were a source of "characters" only.

Now, over time countless people writing countless terminal emulators have tried to address this shortcoming, in countless not-quite compatible ways. And that's been about as successful as you'd expect. You can do it, but...

But again, it's not about device support. Your command line program isn't and has never been connected to a "keyboard" device via any useful abstraction. The same feature that allows you to pipe input into it instead of typing at it makes this problem hard.

agumonkey5• 5 years ago

What is the correct architecture or design to make TUI on a terminal then ? Honest question

TeMPOraL3• 5 years ago

I think the other problem mentioned in the article is that it has to work over SSH. TUI on a local machine has no problem hooking into input devices; display-side, the difference between GUI and TUI is just what API calls you use to render stuff. But if you want to drive things remotely, you're constrained by the remoting protocol - which, in TFAs case, is SSH without X forwarding.

ajross4• 5 years ago

Honest answer: one that doesn't involve the need to capture individual keypresses. It just doesn't work on a Unix tty. Use a web app or other smart client (c.f. the IBM 3270, which had this problem largely solved in 1971 by putting the key navigation intelligence into the terminal).

Athas• 5 years ago

You don't go through the TTY model, but read directly from devices (if the kernel allows it). But people will hate you for it, since it doesn't fit with the abstraction layers that are commonly used, one of which is that only characters matter, not keypresses. Consider how such a TUI that directly interacts with keyboard devices (because it wants to detect keyup events) would work over SSH. It cannot. Abstraction layers limit certain functionality, but they also allow workflows to exist that otherwise could not (or would be much more complicated). You have to judge in every instance whether the correct balance has been made, but I don't think this aspect of the Unix terminal design is awful.

leoc3• 5 years ago

I mean, on Unix it's ncurses, right? If no-one's ever added key-event support to ncurses then that's a(n additional) sign that it likely isn't possible and that TUIs just have to make do without the capability.

mortehu• 5 years ago

There could easily have been escape sequences for released keys, just like there are for non-character keys like the arrow keys. You'll have to change your terminal emulator to support this too, though. This would work fine with SSH.

silon422• 5 years ago

We need a text-mode equivalent of X11 (and then ssh forwarding for it).

arghwhat3• 5 years ago

You're not supposed to write very advanced TUIs (advanced being subjective here). It's not what the interfaces were designed for.

The interfaces for advanced UIs is Wayland and X. TUIs are meant for compatibility, not fancy ability.

toast0• 5 years ago

Like other's have said. If your user interface relies on key-up, it's not really possible in general. A given linux machine may only have a serial console, and afaik, you're not getting key up from any serial terminals; certainly not any common terminals. It's just not part of the design, and short of a time machine, it's unlikely to become part of the design because DEC isn't around to update the spec for vtXXX terminals or their implementations.

zokier3• 5 years ago

Correct architecture for terminal applications is command line; line oriented input, line oriented output. Not handling individual key events.

silicon2401• 5 years ago

this comment was an excellent read. is there any kind of book that covers the historical development of computer tech in this way, that isn't just a full-blown technical reference?

ajross4• 5 years ago

Not at that level of completeness, no. Most people who puzzle this out start with writing escape characters in shell to dazzle up their bash prompts and then escalate from there to figuring out the rest of the protocol.

amelius• 5 years ago

Linux exposes your keyboards as /dev/inputN devices

Yeah, but those are only readable by root.

zozbot2342• 5 years ago

The modern Linux stack allows non-root users to access these if they're "bound" to the right 'seat' and 'session' in the freedesktop-derived multiseat infrastructure. AIUI, this is how non-root X works as well. It's incredibly clunky stuff because no one has ever made the effort to refactor it nicely (the typical problem with all fd.o stuff), but this is supposed to be possible.

fulafel• 5 years ago

To add, you can read the kb events even in a command line program. Using GUI APIs works fine in terminal programs if the GUI is running. Works even remotely in a GUI that support network transparency, like X does. Eg like this:

https://stackoverflow.com/a/4037579

alkonaut• 5 years ago

Detecting keyup only in a window env seems like a perfectly good place to be. The problem there might be that there is no such thing as a "desktop linux" so writing a nice cross platform desktop app for any Linux regardless of window manager and desktop env I assume is a bit of a black art.

josefx• 5 years ago

The X11 example only has a basic event loop with a check if the if the key up event was repeated. Everything else is setting up the window and tearing it down on close. That should work in any window manager.

Most of the examples are either global listeners or outside of a window environment.

marcodiego• 5 years ago

I don't think the article is fair. I'm not used to such lower level on linux, but it is certainly doable without root. SDL has been doing it for ages. Doing it from the terminal is another thing entirely since it was never designed for such kinds of events.

thebeardisred• 5 years ago

I'm right there with you, but as someone who is used to the "lower levels" of Linux (to use your phrasing) this seems like the exploration of someone beginning to understand they were on the question side of an "X / Y problem" (https://xyproblem.info/).

Put differently, they're making the transition from Linux as a commodity used to build the thing they're dreaming of to Linux as a critical design component.

For example, he calls out in the beginning: > They are interested in performing some real-time based task that is controlled using keyboard presses. In my case, the goal was to remotely navigate a robot over an SSH connection using the 'w', 'a', 's', 'd' keys.

As the author discovers, that's fundamentally not how SSH works. This sort of behavior could be achieved using other mechanisms, but it's not even really an issue of the tty/pty. They're just trying to map a functional model from a different operating system to Linux.

As to the discussion of keyboard handling moves into Python code I would have again, gone a different path.

When the author took a turn into Python my first thought was this is trivial for evdev to handle (https://en.m.wikipedia.org/wiki/Evdev / https://python-evdev.readthedocs.io/en/latest/).

Next thing I know they touch on the kernel mechanisms managed by evdev.... And pivot across to X11 (which would seem to make sense until one realizes that the transition to Wayland from X11 is far further along than a layperson might imagine).

In the end a good write up which shows a lot of "raw power" on the part of the author. With some additional tutilage, exploration, or guidance they to really take their understanding to the next level.

For the folks who are in the weeds (like me) it is a good guided tour. Seeing into the "beginners mind" (and taking it to heart) can provide perspective as to how to make software more intuitive.

(Typed and butchered from my phone).

riskable3• 5 years ago

I could be wrong but I'm pretty sure SDL just reads the keydown/keyup events it gets from the X server (that's why you don't need to choose a keyboard device when you use it). So without X running SDL wouldn't be able to grab keyboard events.

It can grab joystick/gamepad events though since it reads those directly. In that case you have to enumerate then specify which gamepad/joystick device to use so it can open the correct /dev/ path.

BoppreH3• 5 years ago

It can certainly be done by interfacing with the X server, but it limits you to systems with a running X server (e.g. no headless raspberry PI's, no Wayland).

I have a planned change to use this a default, and fall back to /dev/input on non-X systems. It's not quite there yet, especially the capability of suppressing key events.

pengaru4• 5 years ago

Userspace doesn't implement debouncing in linux, that's driver territory; linux isn't an rtos. It's unclear why this is even mentioned at all.

And you don't need root if your program's user is in the input group on most distros. You're opening a device node in /dev/input/XXX, the standard UNIX permissions model applies...

zozbot2342• 5 years ago

And you don't need root if your program's user is in the input group on most distros.

It's complicated. Being in the input group is part of it, but to actually get the access you need to interface with the freedesktop-derived multiseat stack, which is as clunky as everything else that comes from the fd.o folks.

klodolph3• 5 years ago

Yes, I think the article's description of getch() as being inadequate due to a lack of debouncing is a bit weird. I would think that debouncing would be done even earlier, in the keyboard itself, since the debouncing you need is necessarily specific to the actual hardware. While USB HID keyboards are state-based (they transmit a packet listing which keys are pressed), the PS/2 protocol is event-based, and transmits "make" and "break" messages for key-press and key-release events, respectively.

(That said, I wouldn't be surprised if debouncing were done in the driver. I just think it's the wrong place for it.)

pengaru4• 5 years ago

I suspect the author may have some experience implementing ad-hoc buttons via GPIO polling from userspace, and maybe used getch() in doing so.

dec0dedab0de• 5 years ago

This might be a dumb question, but why would I want to detect a Keyup event in the first place?

dragontamer5• 5 years ago

Lets say you're porting a fighting game, such as Blazblue, to Linux.

Carl Clover (Blazblue character) attacks on keydown, while Carl's doll attacks on key-up. Expert players change the rhythms of their key-down / key-up to make combos possible.

In general, these kinds of fighting game characters are called 'Negative Edge' characters. There's a large number of them in many different fighting games. I know its present in Street Fighter, Marvel vs Capcom, Mortal Kombat, and Injustice. Even SSB:U (Shield-release / perfect shields) is a negative-edge event, showing just how common negative-edges have become in modern games.

I personally am only really good at Blazblue: and thus, I know that Carl Clover, and to a lesser extent Taokaka and Lichi are negative-edge characters. Its an entire "character design" philosophy, to make certain characters feel much different from others. But its all over the place.

----------

I HATE playing negative edge characters. If I realize a character has negative-edges, I run the heck away from them. Nonetheless, I accept that fighting games are fun because there's "always a character" that matches someone's personality.

If someone else has fun playing negative edge characters, I want to welcome them into the community, and therefore want to support negative-edge gameplay into a game. Its not about "me", its about "the community of players".

theon1442• 5 years ago

The problem isn't Linux, but the terminal environment...

recursive• 5 years ago

Or even simpler. Let's say you're porting a side scrolling old school Mario-type game. When you release the "move right" key, the guy stops moving right.

theon1442• 5 years ago

RTFA

In my case, the goal was to remotely navigate a robot over an SSH connection using the 'w', 'a', 's', 'd' keys. Real-time tasks like this require extremely high responsiveness to key events for palatable performance.

Yeah, I don't think a SSH connection is the right tool either, but there's the answer (and basically the entire article follows from that).

fl0wenol• 5 years ago

In retrospect, thinking about the end problem, he really should have written a custom front-end that used SSH (or TLS) and a custom protocol to remotely send input and receive telemetry, as most term emus don't support this kind of behavior. It could still be a TUI, but it'd be local and use evdev. What'd be nice is then you could actually use a joystick if you had one ^_^

silon422• 5 years ago

To properly implement something like Ctrl+Tab for switching between tabs (assuming application has such a concept).

leoc3• 5 years ago

Seeing this near the top of HN is a bit like seeing a "Why Does Italian Have All These Screwy Verb Endings?" post near the top of a web forum for professional students of linguistics ...

munchbunny• 5 years ago

What's obvious to you might not be obvious to everyone. I've only heard reference to the issue previously but never looked at a detailed discussion, mostly out of not caring because I've never had to do something that depended on detecting keyup events in Linux in a non-browser context.

ajkjk• 5 years ago

Really? There is almost no information in this article that I have ever heard before, and I imagine that I'm a pretty typical reader of this forum. You might have have a very mis-calibrated model of HN readers.

BoppreH3• 5 years ago

I'm the author of the `keyboard`[1] library mentioned in the article.

- It reads all events because it's meant to interface with your keyboard directly, for global hotkeys, macros, disabling or remaping keys, etc. Interacting with windows and per-application hotkeys are explicitly not a goal of the library at the moment.

- It reads /dev/input because the library was developed to work in as many environments as possible, including headless installations like raspberry pi's that may not have a graphical environment or even a monitor. There's an open change to _try_ to communicate with the X server first and fallback to /dev/input, but event suppression is not working reliably with this yet.

- It could read /dev/input by just being in the `input` group, but then `dumpkeys` doesn't work and you are stuck typing numeric scan codes instead of key names.

Due to a series of unfortunate personal circumstances I've been unable to give the proper maintenance the library requires, but the issues and thank yous have never stopped, and I'm slowly getting back to it again.

---

Little known trick: if you run the library as a standalone module (`python -m keyboard`), it prints a JSON object for each detected event. You can save them to a file and pipe them back (`python -m keyboard < events.txt`) to replay them like a macro.

Little known trick 2: I also created `mouse`[2], the companion library for my second favorite peripheral. It's the same thing, but for mouse events.

[1] https://github.com/boppreh/keyboard

[2] https://github.com/boppreh/mouse

RickHull• 5 years ago

For those interested, I wrote a similar library for Ruby: https://github.com/rickhull/device_input

vanous• 5 years ago

Thank you very much for the library. Recently I have been learning touch typing and detecting system wise characters typed per day plus speed and acuracy is on my todo list. I already selected the keyboard module for this after some initial research.

jorl17• 5 years ago

Hi! I've used your library in various projects and am very grateful for it!

I figured it'd be worth a shot to ask you this here, since I just caught you: have you considered adding some support for "rehooking"/"restarting" the library, so that callees can restart your library whenever they detect that their environment has changed (e.g. input devices have been (dis)connected, etc)?

I've contributed with some info to this issue https://github.com/boppreh/keyboard/issues/264 as to how we're dealing with this in some of our projects. Funnily enough, I didn't know about `python -m keyboard`, and had I known about it, it would definitely have saved me some time!

Once again, thank you very much for your hard work!

BoppreH3• 5 years ago

Thanks!

I've been thinking about how to handle environment changes for a while, and it's pretty hard to reliably detect and respond to them in all supported systems.

But I really like the idea of a manual "re-init" method as temporary measure. This should be trivial in Windows, and doable in Linux after some refactoring.

Thanks for the idea, I'll take a look tomorrow.

teknopaul• 5 years ago

This is silly, you can't detect key up or keyb down in a tty, you detect what the terminal emulator gives you: chars and Esc sequences.

It's as sane as asking why you can detect joystick movements or the kettle boiling.

It bothers me how little people understand about the terminal and "the command line", despite using it everyday.

If people read man bash from top to bottom they would probably get a feel for what is going on. Learn how changing the title of your tty works and there is not much left to _not_ understand.

imtringued4• 5 years ago

Yeah, the author is not running into a Linux problem. He is running into an SSH problem.

Something worth noting to avoid confusion is that if you run the example python keyboard example in the introductory paragraph of this article over an SSH connection, the code will still work and run, but when you type characters, nothing will happen.
What gives?!? That's because it will be detecting 'local' keyboard events from the machine you just SSHed into! If it's a cloud server like EC2 or something, there probably aren't any keyboards attached to it!

"What gives?!?"? Is this supposed to be a joke? What else did you expect? Expecting your local keyboard to connect to the remote machine is as insane as expecting your keyboard to read the keys you want to press from your mind.

The problem he is running into is that SSH only forwards a character stream to the remote machine and that is entirely an SSH problem and the solution to forwarding the keyboard has been X forwarding. What he is asking is that SSH should be able to forward the keyboard without X.

He even mentioned a workaround that involves compensating the lack of this feature by forwarding the keyboard himself.

Well, you have to build your own client/server application where a client/server listens locally on the machine with the keyboard, and then forwards these events to the remote machine where you're running the applications that needs to respond to these events.

My suggestion is that he should send an email to the openssh developers to add keyboard forwarding without X.

memoblobi• 5 years ago

Because they are serial character devices?

GuB-42• 5 years ago

Small nitpick, if you use WASD for control, it may be better to use the keycodes and not the characters. People with non-QWERTY layouts probably want to use whatever keys they have on this location (ex: ZQSD for AZERTY layout).

pcwalton• 5 years ago

So, what do you do when you want to send 'key release' events over SSH when the target machine doesn't have an X server? Well, you have to build your own client/server application where a client/server listens locally on the machine with the keyboard, and then forwards these events to the remote machine where you're running the applications that needs to respond to these events. That sounds like a lot of work because you have to set up all the sockets and custom messaging protocols, but there's no way around it because the key release events simply aren't forwarded over your terminal-based SSH session when there is no X server forwarding configured.

This isn't that much work if you write it as a trivial Web app using WebSockets. Then you can leverage the browser's OS abstraction layer instead of having to write your own.

account42• 5 years ago

Are you interested in detecting local 'key up' events over an SSH connection without involving an X server? Oh boy, are you in for a disappointment! It turns out, that it's impossible. I don't mean the "I tried really hard and couldn't figure out how to do it, so it must therefore be impossible" kind of impossible, but the real kind of impossible where the keyup event doesn't get communicated at all during a non-X forwarded SSH session. Here is an experiment you can do to prove this to yourself [...]

This isn't really a good experiment as it assumes that everything that can be sent over SSH will always be sent. This not the case in the terminal world where by default you get a processed character stream and have to explicitly enable anything else. For example kitty's keyboard protocol [0] mentioned elsewhere in this thread needs to be activated by sending the escape sequence CSI > 1 u to the terminal on the other side of the SSH connection.

[0] https://sw.kovidgoyal.net/kitty/keyboard-protocol.html

Why Is It So Hard to Detect Keyup Event on Linux?

Related Stories