QtWayland 6.6 Brings Robustness Through Compositor Handoffs

http://blog.davidedmundson.co.uk/blog/qt6_wayland_robustness/

122 60 3 years ago

blog.davidedmundson.co.uk

jangid• 3 years ago

Could this potentially open up the possibility of implementing a user-level desktop session serialization and restoration feature? Similar to how Emacs handles per-directory sessions.

p4bl0• 3 years ago

It seems to be what they're going for at KDE:

https://floss.social/@kde/111051338968313784

Plasma developer David Edmundson demonstrates how a desktop using Wayland, Qt6 and KWin can recover from a catastrophic crash as if nothing had happened.
You will lose no data, the video you were watching will not skip a frame, and the contents of your clipboard will remain intact.
The same principles can be applied to jumping from one desktop environment to another, for example, from Plasma to Gnome...
... And can provide a way to save _the state_ of an application to disk, stopping the app in its tracks and removing it from memory, so that later you can restore it just where you left off.

sho_hn6• 3 years ago

This is more or less what we once dreamed of doing with Activities in KDE 4, but couldn't make it work on X11 - we were banking on the X11 session handling protocols to implement the suspend/restore, only to find that few if any toolkits and apps implemented them properly, and that fixing this wasn't going to be viable due to a lot of spaghetti all over the place.

This is partly why Activities ended up feeling somewhat redundant to Virtual Desktops. But if you go back to those early 4.x releases, you will find that the Pause/Unpause buttons etc. on Activities were featured rather more prominently.

As David describes in the blog post, things in Wayland are a lot more nicely layered. In part, toolkits have also seen architecture cleanup as a side effect of having to support multiple backends during the transition, and code has become more hackable and modular as a result.

c-hendricks• 3 years ago

Finally, save states for Minesweeper.

(This is genuinely interesting, I'm not sure how practical it is, but still very cool.)

saidinesh5• 3 years ago

I think the really practical part of this is dealing with out of memory use cases. Iirc he mentioned that in the video too.

CoastalCoder• 3 years ago

Could someone explain the significance of this for Wayland adoption?

As an outsider to the X11 vs. Wayland discussions, my impression has been that the main barriers to simply ditching X11 have been:

(1) poor Wayland support in nVidia's proprietary drivers

(2) Wayland's security model making some X11 use cases, e.g. screen recording, difficult or impossible.

Does QtWayland 6.6 address either of those (and/or some other) barriers?

traverseda3• 3 years ago

Well KDE has been pretty good about implementing wayland extensions for things like screen recording, accessibility, etc. Gnome has also been implementing extensions but they tend to go off and do their own thing.

The main barrier is honestly expecting a bunch of unpaid open source developers to go and re-implement everything. Stuff like barrier/synergy technically has the extensions needed to add in wayland support but it's still all unpaid volunteers.

There also used to be some more leeway about protocols and extensions. For a long time now Gnome has been saying "you're either a gnome app or you're not" when they deprecated stuff like tray icons. But there was generally a way back, a way to run your non-gnome app cleanly on Gnome. With the introduction of wayland they seem to be more set on forcing developers to choose, like they really are blocking tray icons now. There are common desktop extensions that Gnome just isn't really interested in developing.

tristan957• 3 years ago

unpaid volunteers

This is not entirely true. A few GNOME developers within GTK/Mutter/GNOME Shell are paid to work on this kind of work at least part-time. SourceHut also pays Simon Ser to do some Wayland work, whether that is maintaining wayland-protocols or something else.

With the introduction of wayland they seem to be more set on forcing developers to choose, like they really are blocking tray icons now.

This is also not true. GNOME has designs for tray icons in their GitLab repo. The thing that is blocking better tray icons in Linux is a lack of interest in finishing the protocol proposal. The ticket hasn't seen much traction in the last couple of months.

https://gitlab.freedesktop.org/xdg/xdg-specs/-/merge_request...

hulitu4• 3 years ago

expecting a bunch of unpaid open source developers to go and re-implement everything

We are really wondering why those "unpaid open source developers" would want to "re-implement everything". Fixing existing systems is hard, i presume. /s

type0• 3 years ago

There are common desktop extensions that Gnome just isn't really interested in developing.

In GNOME 45 all current extensions will loose backward compatibility, that's one of the yet more changes that will force many to migrate to other desktop environments

sprash7• 3 years ago

You forgot the most important reason:

(0) zero community mind share as a result of extremely cumbersome and developer hostile APIs and the lack of standardization of essential functionality.

QtWayland won't be able to address this because vendor lock-in for a de-facto proprietary toolkit won't generate any community mind share either.

sho_hn6• 3 years ago

(1) poor Wayland support in nVidia's proprietary drivers

This has improved quite a bit with nVidia eventually implementing GBM support (so compositors no longer need an nVidia-specific codepath, which is nice). I use Plasma on Wayland on my nVidia-powered ThinkPad, and it's generally fine and has been for some time. That said, the nVidia stack does still feel a bit more hit and miss than some others.

(2) Wayland's security model making some X11 use cases, e.g. screen recording, difficult or impossible.

These largely have been addressed by newer protocols and infra, although application support and maturity for those protocols in deployed systems is still trailing behind X11. Still, the fundamentals have been worked out, and there's steady progress on spreading the solutions through the ecosystem.

And no, this blog post has nothing to do with either topics but is about unrelated technical innovation in the Wayland space.

FloatArtifact• 3 years ago

Accessibility software needs access to the window details like executable, new title, window size and so forth and to be able to emulate peripherals.

tov_objorkin• 3 years ago

One very stupid reason is the Qt itself, graphics application can't live without the QScreen instance. If user unplug every screen from the system, in order to prevent crash qt create the fake screen and hang in the headless state.

pseudotrash• 3 years ago

Nitpick: there is no "vs." ... because in the long run X11 is dead. After decades of serving us well it's not maintainable and not a good base to build anything on top that you want to continue maintaining in 5 or 7 years. Any warranty of a "cyber physical systems" is better off starting on Wayland IMHO

What issues are you facing with screen recording?

Can't say anything about NVIDIA because I avoid these chipsets like the plague (even for windows)

hulitu4• 3 years ago

X11 is dead. After decades of serving us well it's not maintainable

Why not ? Is the wayland "maintainable " ?

nsajko• 3 years ago

This is just FUD.

dekhn• 3 years ago

X11 will continue to exist and be maintained for decades. I would not even be surprised if it lasts longer than Wayland.

umanwizard• 3 years ago

I am still using X and it is still working fine for what I need. Why would I need it to continue to be maintained?

PurpleRamen• 3 years ago

[..] Compositors and displays servers are now the same process, doubling the space for errors

The wayland security model means the compositor absorbs even more functions from global shortcuts to screencasting and input-method handling [..]

Doesnt that mean, Wayland becomes everything that X11 was, just worse? I thought, Wayland was created to break up the monolithic X-Server?

sprash7• 3 years ago

The severely limited scope of Wayland forces all DE vendors to reinvent the wheel and basically recreate their own X11. But this time it's worse because the APIs are DE/Toolkit specific without the standardization that X11 offered.

Also X11 was never "monolithic" but in reality completely modular. It allows for example to change the window manager or even compositor at runtime without affecting running programs. For this to work your API hast to provide more functionality than your typical Wayland compositor. This is often mistaken for "monolithic" when in fact is actually the complete opposite of monolithic.

jauntywundrkind• 3 years ago

That's a typical view, but I see it as a lot less monolithic because it's using the kernel to do a huge amount of the things X used to do itself.

There's a lot of other concerns (input handling, screen shots, etc cetera) but at the heart Wayland is mostly just a management later for *nix dmabuf's. Mode setting concerns are done by the kernel too. Libwayland mostly uses egl, is my understanding. It invents so much less than X, where-as when X was made, the OS had none of these abstractions to reply on. X was basically the kernel, part 2. Wayland has none of this baggage; in almost all implementations leverages a world of great ready made stuff rather than implementing itself. Source: https://wayland-book.com/surfaces/dmabuf.html

kaba02• 3 years ago

to reinvent the wheel

Libraries are a thing

because the APIs are DE/Toolkit specific

That’s false in itself, what do you think Wayland protocols are? They might add some DE-specific protocol for themselves, but I don’t see why that would be a problem.

standardization that X11 offered

What standardization? A single implementation that does its thing is not a standard, by definition.

Also X11 was never "monolithic" but in reality completely modular

But let me guess, you think that systemd is a huge monolith, right? Also, a wm is very trivial compared to the rest, they can easily be a lua extension and that’s it.

X is monolithic, because they had to pry printing and shit out of it over years of hard work.

sho_hn6• 3 years ago

But this time it's worse because the APIs are DE/Toolkit specific without the standardization that X11 offered.

On this one: It's not really that different. Very little if any KDE code made any direct calls to Xlib or xcb past the very early years. Most of this was also hidden by Qt abstractions, and toward app/shell devs most certainly by further KDE library abstractions. Directly interacting with X11 was an almost-never occurence.

Fairly similar in other toolkits, since most of them have some sort of cross-platform strategy and infra.

People sometimes have kind of odd notions of how exactly X11 development was different. X11 had its fair share of "fragmented implementations" immaturity. It had its fair share of "this new standard is not widely implemented yet" phases that people just don't remember because they started using it after things had settled down (e.g. NetWM and EMWH protocol extensions did not always exist, and also got interpreted quite differently sometimes, or e.g. the clipboard spec). DEs also did plenty of DE-specific things on X11 using proprietary window hints or DBus-based side channel protocols.

If there's one broad philosophical difference, it's Wayland has some widely-respected values around who gets to introspect what state and who acts as authority, that mean that things that X11 apps could monkey-patch may now require a protocol to be agreed on. But there's a lot of value in agreed-upon protocols, and X11 also needed many of them.

agildehaus• 3 years ago

Wayland sounds a bit like what Vulkan is to OpenGL? Am I anywhere close?

I'm a novice when it comes to the implementation of compositors and display servers, but as a user it's been a far better experience than X11/Xorg ever was. Seeing this robustness video just proves it to me even more.

kaba02• 3 years ago

What’s the difference between a display server managing frames from applications for display, and compositors.. managing frames from applications for display but possibly with fire and wobbling? That’s like a bad place to draw a line.

There is no problem with monoliths in itself, if you needlessly introduce IPC at boundaries that don’t need it, you make your life just harder (also, tell that to those who make everything into a microservice). Also, people can use libraries to create better, different implementations, e.g. see wlroots - isn’t that the spirit of open-source instead of a single de facto monolith?

The important thing is that the public API is well-defined.

Also, X has problems and needs replacing because its fundamental architecture got outdated in the last 30 years.

josefx• 3 years ago

clients relied on memory stored by the Xserver, they made synchronous calls that were expected to return values, and multiple clients talked to multple clients.

Wasn't that just an issue with the Xlib interface? I thought Xcb made everything async.

vidarh2• 3 years ago

Yeah, very little in the protocol is sync. E.g. when you create a window the client picks the id, and you just go on assuming it will be created until/unless an error arrives.

Only things explicitly querying for information is "somewhat" sync, but then too the reply will arrive back with a sequence number, so the client API can just keep returning events and hold on to the replies until you ask for them.

X has many warts, but it being synchronous isn't one of them.

hulitu4• 3 years ago

X has many warts, but it being synchronous isn't one of them.

I seem to remember that there was a command line switch to run X (or some clients) in syncronous mode. It was mostly used for debugging.

renox• 3 years ago

This reminds me of Arcan where the goal number 1 is robustness, which is probably a good idea given the complexity of GUIs.

soupbowl• 3 years ago

Arcan has some really cool ideas.

throwaway1984s• 3 years ago

https://arcan-fe.com/2017/12/24/crash-resilient-wayland-comp...

6 years ago and no toolkit modifications required.

sho_hn6• 3 years ago

Apples vs. oranges, though. If I read the blog post right, he's demonstrating that he can restart the main Arcan process while an auxiliary process called "waybridge" lives on, which for the Wayland clients is the actual compositor. This still requires the same type of "reconnect and restore" dance, but internal to Arcan-proprietary bits, pushing the problem up one level from the native Wayland exchange.

The blog post being discussed here is about handling a crash in "waybridge" instead, so to speak, and having this ability in Wayland itself instead of requiring an additional abstraction and protocol.

Arcan can use multiple waybridge instances, e.g. one per client, to get some isolation between the clients (from the post), but it does start to sound a bit heavy perhaps (I don't know much about the inter-Arcan IPC though).

Cool stuff, though.

QtWayland 6.6 Brings Robustness Through Compositor Handoffs

Related Stories