Moving the Linux desktop to another reality

https://www.collabora.com/news-and-blog/news-and-events/moving-the-linux-desktop-to-another-reality.html

by mfilion • 7 years ago

175 93 7 years ago

One of the authors of Simula[1] here (mentioned in this article).

xrdesktop looks very impressive. We've been working on our compositor (in particular: XWayland support) for quite a while. Getting a prototype up and running was relatively easy. But it has been incredibly difficult to get everything actually stable/usable.

Congrats to your team.

[1] https://github.com/SimulaVR/Simula

echelon2• 7 years ago

Thanks so much for working on this!

I really like multi-monitor setups, and in the limit this can far surpass physical hardware and space limitations. You can scroll more than 360 degrees for a near-infinite workspace. (Just spin around a hyper-sphere repeatedly.)

I really want to see this tech succeed. I'd look like an idiot using this at work, but I'm already sold.

I'm forced to use a Mac at work -- do you know of similar tech for MacOS? I'm gladly going to investigate this at home.

georgewsinger2• 7 years ago

Thanks. This gives us motivation to keep going. If you have any feature suggestions or want to check on our progress, drop by our chatroom in discord (https://discordapp.com/channels/603723949586644997/603723949....

I don't know of any similar tech for MacOS at this time.

skrowl• 7 years ago

I really wish the "Yay, linux on the desktop" crowd would rally around ChromeOS. ChromeOS is literally what the "linux on the desktop" crowd has been wanted for decades now.

Zak5• 7 years ago

Heavily integrated with proprietary services is definitely not what the "linux on the desktop" crowd wants.

axaxs• 7 years ago

What...why? I'm perfectly happy using linux on the desktop as my full time OS, as I have for more than a decade. I tried ChromeOS on chromebooks, and see nothing they offer me that I don't already have. Not to mention the privacy implications of using such an OS...

AdmiralAsshat• 7 years ago

The vast majority of Chromebooks are completely useless without Google services. That's not exactly the "freedom" that the Linux community envisioned when they spoke about "The Year of Linux on the Desktop".

dflock3• 7 years ago

Well, you can install regular linux and run whatever you want: https://support.google.com/chromebook/answer/9145439?hl=en

FigBug• 7 years ago

Can you run a 'normal' Linux desktop application written in C or C++ on a Chromebook? Or is it just Android Apps and Web Apps?

dflock3• 7 years ago

Yes, you can: https://support.google.com/chromebook/answer/9145439?hl=en

dreamcompiler3• 7 years ago

You certainly can if you install Linux on your Chromebook. I've been using my Chromebook this way for years. But I use a real Linux distro independent of Google (my system will not run Android apps). I have no idea if Google's relatively recent Linux offering works as well.

asark3• 7 years ago

Chrome OS doesn't give me the benefits that drive me to sometimes use the Linux desktop, so it doesn't really matter whether it addresses any of the (many, serious) problems that keep me from using Linux for everything.

dflock3• 7 years ago

You can run real linux as well a ChromeOS: https://support.google.com/chromebook/answer/9145439?hl=en

Zak5• 7 years ago

I've been wanting the option of a VR desktop for a while now. Last I looked, it wasn't ready for prime time, and that's still my impression now. I find a small 16:9 screen pretty unacceptable for coding, and the market currently offers very limited options for anything taller in a laptop.

I'm eager to see where this goes, and may need to start shopping for a head-mounted display.

gmueckl9• 7 years ago

The current generation of headsets is unsuited for showing large amounts of reasonably sized text. The display resolution is not there yet. If you map your desktop into a plane in VR space at the proper distance, your UI text shows up heavily aliased and barely readable due to the texture resampling that happens in that process. You need to increase resolution by another factor of two or three to make that actually convenient.

And then there is the issue with holding 3d pointing devices for prolonged amounts of time. This is taxing upper body strength if you do. The weight of thenheadset itself is also straining the neck muscles.

All of this taken together usually means that I want out of VR after about 2 hours. I barely have sessions that are longer.

Zak5• 7 years ago

What's the resolution of the headset you've been using?

I don't think I'd like my desktop to have a lot of 3D features or require a 3D pointing device. What I'm imagining is essentially a giant 2D workspace where I can still use a Trackpoint.

asark3• 7 years ago

And then there is the issue with holding 3d pointing devices for prolonged amounts of time. This is taxing upper body strength if you do. The weight of the headset itself is also straining the neck muscles.

You could use a real mouse and keyboard, if you're presented with some kind of virtual workstation. And even for navigation outside the workstation, of your broader "environment", 3D video games have been using a mouse & keyboard to navigate complex interfaces & environments for a long time, with more than a little success. Granted that means you still need a surface of some sort, but I think non-touch-typing text input inside a VR space would be kinda hellish anyway, so you're at least gonna want a real keyboard regardless.

dreamcompiler3• 7 years ago

Lighter, higher-resolution headsets + full keyboard control = 3D Emacs!

erikpukinskis• 7 years ago

The difference between a headset and a monitor is that in VR when you lean in the text gets sharper.

derekp7• 7 years ago

I care more about vertical resolution, and vertical height, more than about aspect ratio. So with 3k and 4k getting more popular on higher end laptops, the 16:9 doesn't bother me as much. I can keep a code window on one side and documentation (or program output) on the other side of the screen.

The only other drawback of 16:9 is that in order to get an acceptable height, it makes the laptop too wide. My solution is to use clip-on reading glasses over my regular distance prescription, when using a high res laptop screen.

Zak5• 7 years ago

I can imagine that solution working, but what I'd really like to see is some serious business laptops offering 3:2 screens, or 16:12 for that matter. The screen from the large iPad Pro would be perfect on a laptop.

choeger• 7 years ago

That page crashes my Firefox mobile... Is that some kind of inside joke I don't get?

davesmith19833• 7 years ago

This is going to be another compiz cube. It looks cool, but I doubt it will be very useful outside of showing off some cool tech.

Also Valve tbh have dropped a lot of their more interesting tech. I had one of their steam links which isn't even sold anymore. The steam link was actually pretty useful for steaming anything on your desktop, I am sure you can do the same with a Raspberry Pi and some scripts but it worked pretty well if you hooked up a mouse and keyboard.

e1ven• 7 years ago

Not to go too far off-topic, but if you haven't tried them -

I've found that the steam link apps for other platforms (ios/android) work very well. I'm not sure Valve needs dedicated hardware for this anymore.

I have a steamlink in the closet, but being able to do it in software is nice.

davesmith19833• 7 years ago

If you have steam installed on anything you can just stream to that device as long as they are on the same network.

larrik2• 7 years ago

I love the Link, I use it all the time. I even bought a spare.

logosmonkey• 7 years ago

Yeah, I bought an extra one when they discounted them to near nothing after they announced they were discontinuing them. It's nice to be able to play games on my bedroom TV when I'm going to bed :)

skykooler• 7 years ago

Compiz cube was what convinced me to try Linux in the first place - I had always thought of it as a boring server OS, but then I saw the desktop cube and wobbly windows and thought "that looks cool enough to try it out." Hopefully this could have a similar result for some people who have VR headsets.

babarock• 7 years ago

Same. And I like many, I turned that damn cube off after 2-3 months of use.

jimktrains2• 7 years ago

I _loved_ the wobbly windows! But not enough to figure out how to reënable them in my most recent install ️

rocky11382• 7 years ago

Same! I used it for years. I loved the idea of having different desktops on each face of the cube. Compiz was so good.

tom169• 7 years ago

Same!

groovybits• 7 years ago

I am sure you can do the same with a Raspberry Pi and some scripts

Actually you don't even need to write your own scripts. There is an official steamlink package available for Raspbian. I just set up a Pi3 to run as a dedicated steamlink the other day, and it works fairly well.

Just run:

  apt install steamlink

  steamlink

Edit: And if you want it to run the steamlink interface on bootup, you can stick this in your crontab:

  @reboot steamlink

davesmith19833• 7 years ago

Thanks for the info. I wasn't surprised in the least that there was something out there. I am a nomadic dev at the moment so most of my kit is locked in a warehouse and I am stuck using MacBook pro.

giancarlostoro• 7 years ago

Nice, I was curious what repo it was in since it doesn't seem to be in Debian Unstable anymore.

Anyone interested can get the Debian package here:

http://archive.raspberrypi.org/debian/pool/main/s/steamlink/

wlesieutre2• 7 years ago

I don’t know that most people would use this, but there’s a definite benefit that a VR headset can give you a grid of six 40” monitors for $400. Or whatever display setup you want. It’s an economical way to get a lot of real estate.

The resolution kills that notion for now, but someday it might become a cost effective setup.

Add a pair of noise canceling headphones and you can pretend your tiny desk is a private office! It even has imaginary walls to hang things on!

drudru11• 7 years ago

Totally agree. Tried this on an Oculus GO. Didn’t work well. Not enough resolution and the dev envy leaves a lot to be desired.

smcnally• 7 years ago

How far off is current headset resolution from what’s needed for reading code, working with files, audio, video?

snek• 7 years ago

Goodness this website is slow

amfsn• 7 years ago

https://www.collabora.com/assets/images/blog/xrdesktop/xrdes...

Oh god... where's your anisotropic filtering? That looks like a screenshot from Doom.

flyingfences• 7 years ago

Some antialiasing wouldn't go amiss, either...

oldgun• 7 years ago

In case you experience slow loading of this hackernews'ed website, here's the archive: https://web.archive.org/web/20190730153132/https://www.colla...

canada_dry• 7 years ago

I have a VR setup and would love a fully immersive desktop environment to develop/program in. The display quality of the new Index (for instance) is sufficient... however, the critical show-stopper is not being able to use your hands to type onto a real keyboard (i.e. augmented reality-ish).

A real keyboard that you can feel and see as you type on. Both hands would need to be tracked (in addition to the keyboard location) using something like leap motion not by wearing some gloves/controllers or attaching anything - no one is going to do any amount of typing on a virtual keyboard.

When there's a good solution to this I suspect quite a few developers will give it a go when they want a fully immersive in-the-zone experience.

Wowfunhappy3• 7 years ago

Once your hands are on the keys, you don't actually need to see them. So the real question is, how do you find your keyboard in VR?

I think the Index's cameras would be sufficient. Alternately, you could place tracking pucks (or just controllers) on either side.

shultays2• 7 years ago

Your real keyboard can have a virtual representation in your VR space, just like controllers are

mncharity7• 7 years ago

The display quality of the new Index (for instance) is sufficient...

Is it? Are the lenses that much better? My Lenovo Explorer WMR is 1440^2 px/eye, and thus has a similar angular resolution to the wider FoV Index. Both are RGB, not Pentile. But I find each eye gets a central circular region something vaguely like 300 px diameter where subpixel rendering is worthwhile (I've a custom render stack), and less than 600 px before the lens blurs pixels together. That's 1980's VGA-like resolution. And it gets even worse as objects move in from infinity and thus out of these regions. If you disable the "oh, its an HMD, not a normal display" feature the OP speaks of, you can easily throw up a pixel-aligned test pattern image. The results are... blurry.

So I use DJI's 1080p drone goggles. Readable 5 pt fonts, almost corner to corner. With shutter glasses, because DJI didn't expose the two panel feeds. :/ And reading glasses, because DJI's lenses... sigh. It's a placeholder kludge.

Nreal's 1080p AR glasses could be available in a couple of months.

the critical show-stopper is not being able to use your hands to type onto a real keyboard

I've been using my unremarkable laptop keyboard since Vive, by doing pass-through AR with a camera gaff-taped to the HMD. With a browser-as-compositor stack, it's two-video-dom-elements trivial. Though I've heard with a SteamVR stack, pass-through latency can be a challenge.

Hand tracking is more problematic. It's a poster child for acquisitions wiping out market infrastructure. The sole hand-tracking survivor was Leap Motion, which fumbled its couple of Apple acquisition attempts. It's been Windows-only, stalled, kind of crufty, and now has finally been acquired, with uncertain consequences for future availability. And one can kludge. But it isn't pretty.

When there's a good solution to this I suspect quite a few developers will give it a go when they want a fully immersive in-the-zone experience.

And shallow 3D seems to have interesting possibilities for IDEs.

Wowfunhappy3• 7 years ago

Is it? Are the lenses that much better?

Yes, the lenses on the Index really are that much better. The difference is enormous.

I'm still not sure the resolution is quite high enough that I'd want to use a VR desktop environment, but I absolutely could now, reasonably comfortably.

catotheyoungest2• 7 years ago

Why do people think this worth trying to implement? Is it because it looked cool in movies like Minority Report? What is the target demographic for this? What is the use case? Why would I want to add additional and unnecessary complexity to my workflow?

just_myles• 7 years ago

1. It looks cool. 2. Minority report, no. Gits, yes. 3. Everybody. Probably nobody. 4. Probably to multi-task. Multiple screens that sort of thing. VR gaming had the same problem in the beginning. Wasn't until games like adrift and beat sabre showed up that it started to finally hit a stride. Or at least a compelling use case (Good VR games to play.). 5. See 1.

rocky11382• 7 years ago

Have you tried Microsoft HoloLens, Virtual Desktop, or BigScreen? It was obvious to me after using them for 30-60 minutes that this is the future of desktops.

catotheyoungest2• 7 years ago

I did. Then I spent the weekend with a migraine. IMO, VR isn't worth the effort. I'd rather have my brain directly jacked into the computer, as long as there's an effective firewall available to keep advertising and other malware out of my head.

shadoxx• 7 years ago

Once VR gets to a certain point, and the resolution becomes so undeniably crisp, we'll want things exactly like this. You can already do this on Windows with Virtual Desktop Streamer and a companion application for either the Go or the Quest (paid application). If you can touch type, it's very easy to strap on the Quest and use it as your primary monitor. Having had this experience, I'm guessing that we're only one or two generations of hardware away from this being a normal way of computing. You know, provided smartphones, tablets, and otherwise proprietary operating systems don't take over first, Brother Oculus included.

echelon2• 7 years ago

I want this right now.

I use four monitors at work and two 4K monitors at home. A high-resolution VR setup gives you infinite workspace. Couple that with intelligent auto window management and you can forget moving windows around EVER again -- this is the ultimate mouseless experience that a tiling window manager can't even get close to.

This is a huge productivity gain. The micro context switches that stem from moving between windows and tabs (or worse, virtual desktops each with windows and tabs) goes away. Workspaces become spatial, and you can leverage your brain's innate spatial reasoning to find and organize things.

Physical project workspaces are laid out logically. We've been constrained to screens and haven't had the freedom of doing this for our virtual ones.

travbrack• 7 years ago

Have you done a lot of VR gaming? My eyes are killing me after an hour. I can't imagine doing 8 hours a day. Not saying this everyone's experience, and I'm curious if others are more tolerant of spending long periods of time in VR.

shultays2• 7 years ago

VR resolution drops very quick when you look at something virtual from half meter away. If your virtual screen takes up to 1/4th of your screen for example it will be reduced to ~720p in a vr with 4k resolutipn per eye. Virtual desktops are not really useful as you think.

Plus VR strains your eye really quick and that comfortable to wear for long periods of time

DonHopkins• 7 years ago

This is just like when motion pictures were first invented, and all people could think of to do was to point a camera at a stage and record a play in one continuous shot.

JoshTriplett• 7 years ago

Looking at the screenshot of Wikipedia rendered on an angled window, this really needs sharper text rendering with https://blog.mozvr.com/pathfinder-a-first-look/ or similar.

With something like that integrated into rendering libraries, I'd love to have the field-of-view of a motion-sensitive environment. (I don't actually want pseudo-3D, just a high-res 2D display attached to orientation sensors.)

aiscapehumanity• 7 years ago

I can sort of see this as an incentivizing stepping stone through generally alternate 3d interfaces to to really get at AR desktops, which could be used in conjunction keyboads and gestures which is alot more viable to even code with/in.

Moving the Linux desktop to another reality

Related Stories