Stereokit – Open-source mixed reality library by Microsoft

by yolo123 • 4 years ago

130 47 4 years ago

trzy• 4 years ago

The time has never been better for a new “game engine” or 3D app framework. Unity is trying to be everything for everyone and hasn’t done a great job of taking care of its core audience: indie game developers.

Unity is the de facto choice for XR developers but it’s a poor application framework. Something like StereoKit that caters to non-game devs and can provide rendering, a flexible interaction model, UI, and a good multi-user app model could really steal Unity’s thunder for multi-platform development.

koujaku14• 4 years ago

StereoKit started around 3 years ago, when the first pieces of OpenXR started coming out! OpenXR's native experience really showed just how good XR development _could_ be, and so we built StereoKit around that :)

And then yeah, if you're building apps, and not making a game engine, there's a lot of things you decide differently about the architecture! Like StereoKit's asset pipeline works at runtime, with user-space formats like GLTF. You can load up StereoKit's built-in file picker, and bring in an image or 3D model with 3 or 4 lines of code! Or grab it from a REST API, whatever.

matrixcubed• 4 years ago

I’d recommend you take a look at Godot. With in-built VR and C# support, a new major version peeking over the horizon, and many, many Unity devs coming over due to recent fallout in those circles, the time truly has never been better.

kensai• 4 years ago

I wonder if it can be embedded in the future in Unity.

koujaku14• 4 years ago

StereoKit is an alternative to Unity that skips all the bloat, doesn't really work in concert with it. You could take some inspiration from StereoKit, and maybe port some code snippets, but you're better off using something like the MRTK if you're working in Unity!

jonhohle3• 4 years ago

Somewhat meta, but why would Microsoft I use the Kit suffix which has been widely used by Apple adopted from NeXT?

koujaku14• 4 years ago

That'd be my fault, me and my co-workers were working with a lot of tools that had the "Kit" suffix when we started on it :O Apple's definitely not the only one who does it!

wutwutwutwut4• 4 years ago

Because the term is not allocated specifically to Apple.

jonhohle3• 4 years ago

Sure, but it has a 30 year lineage in their active product line. It would be like prefixing a library with Direct in the early 00s. It seems ignorant and derivative.

tokinonagare• 4 years ago

For the same reason they registered the domain typescriptlang.org instead of typescript.org: riding on the hype train created by others.

Besides, the project looks really interesting, I'll have a look into it.

wutwutwutwut4• 4 years ago

What is the hype train you associate with typescriptlang.org? "Lang"?

evan_• 4 years ago

I think they probably used typescriptlang.org because typescript.org was registered in 2006 for a CMS called Typescript.

ramesh312• 4 years ago

Somewhat meta, but why would Microsoft I use the Kit suffix which has been widely used by Apple adopted from NeXT?

The *Kit libraries from Apple are always capitalized. UIKit, SceneKit, WebKit, etc. The generic use of *kit for other libraries is pretty common though.

flohofwoe• 4 years ago

Kudos for the Zig and V examples (or rather example skeletons), how very un-Microsoft-like to venture into such bleeding edge territory ;)

(the actually great thing is that the engine exposes a plain C API: https://github.com/StereoKit/StereoKit/blob/master/StereoKit...)

PS: double-kudos for making the API C99 designated-init-friendly, at least as far I can see from skimming over the source (e.g. "zero means default": https://github.com/StereoKit/StereoKit/blob/1bf86b281e883041...)

koujaku14• 4 years ago

Thanks! Your work has actually been quite an inspiration for me personally, I've definitely picked up a few of your tricks here and there :)

Language bindings have always been a part of the long-term plan, C# is just the golden path we've optimized for! A good number of people do also use the C API, and there's some community driven effort to create rust bindings as well.

creativenolo3• 4 years ago

StereoKit seems like straight up cut to the fun. In my little free time all the other VR & MR platforms have so much cruft. Unity, endless configuration. Unreal, clone an fork of it and build it. Please I want to play & occasionally teach.

Any plans for AR support?

koujaku14• 4 years ago

StereoKit was initially created with HoloLens 2 in mind, so it does plenty of AR! Also works on a North Star on Linux, as well as Varjo and Quest's passthrough AR. The only catch there is it doesn't do phone based AR like AR Core/Kit. Maybe someday, but for now StereoKit is focused on OpenXR compatible devices.

creativenolo3• 4 years ago

Thanks for the reply (and great work on Stereokit)

I was bucketing HL, Quest pass through, etc as MR and AR as being phone/screen based.

For example, I have been impressed by MRTK's AR support as a potential XR(?) framework (but Unity and the setup is the chore for experimentation)

Hypothetical on the "someday", are OpenXR and WebXR related? As in, could WebXR enable StereoKit to be AR "someday" before ARCore/Kit?

dTal• 4 years ago

I can recommend LÖVR as another "cut to the fun" VR framework. It's even easier to work with than StereoKit; no dependencies, just a binary. You program it in Lua.

https://lovr.org/

lostmsu3• 4 years ago

I see font rendering is not adapted to 3D in the screenshot, so it is not yet ready for serious UI work. I presume curves are not rendered in an adapted way either, so everything will look pixelated around the edges.

koujaku14• 4 years ago

StereoKit doesn't currently do SDF font rendering yet, no, but there is some shader work done to help make existing text a bit clearer than the naive case. It does support Unicode font rendering though, font glyphs are added to the font atlas on-demand to ensure that's a smooth and automatic experience!

The current implementation of SDF conversion is just a tad too slow to make on-demand glyph rendering comfortable right now, so there's a bit of additional optimization work that needs done before we can switch to that. We were just prioritizing language support over text clarity to start with.

wutwutwutwut4• 4 years ago

Is it not possible to do serious UI work without fonts adapted to 3D?

koujaku14• 4 years ago

I would say that the majority of people don't notice StereoKit uses regular raster for font rendering! However, that number is definitely not zero, and there's definitely people that can spot it right away, as evidenced. Designers in particular would probably spot it right away.

I'd say it's primarily a question of polish and feel! Those that can't spot it right away would still prefer SDF fonts in a side-by-side, and the difference becomes much more clear if you start getting your head really close to the text. I consider it to be an essential feature for a 1.0 version, but not a blocker for right now.

Vermeulen• 4 years ago

Stereokit is fantastic. Ive been looking for a .net game engine to try - and love the runtime asset importing.

Are examples of its use for a traditional desktop use? Like screen space UI. And maybe mobile touch input? Or are both of those out of scope and XR is the focus. I am wanting to try a side project in it - but hoping to make it both VR+Desktop.

koujaku14• 4 years ago

StereoKit very much prioritizes XR, I don't think I'd recommend doing traditional flat or hybrid XR/flat experiences with it! You might be able to do some flat 3d content, but the built-in UI won't be very helpful in this environment.

verdverm2• 4 years ago

How does this relate to or overlap with Microsoft MRTK? https://github.com/microsoft/MixedRealityToolkit-Unity

(@koujaku)

koujaku14• 4 years ago

StereoKit and MRTK are currently part of the same team, so these people are my co-workers! :) We swap techniques, code, share design docs, etc. StereoKit joined the MR Middleware team towards the start of this year, and over time you'll likely see StereoKit move much closer to the Microsoft MRDL (Mixed Reality Design Language) and pick up similar features.

jayzhan• 4 years ago

why build two toolkits that are similar?

johnhenry• 4 years ago

https://bobsutton.typepad.com/my_weblog/2012/07/dysfunctiona... (Historical Microsoft)

asiachick• 4 years ago

It's depressing how there's so little VR for Apple. Maybe they'll blow us all way with the iPhone of VR soon but at the moment my Macs have no VR love.

To make this related, I was prompted to this thought by the fact that the list of supported platforms does not include macOS

ramesh312• 4 years ago

It's depressing how there's so little VR for Apple. Maybe they'll blow us all way with the iPhone of VR soon but at the moment my Macs have no VR love.

There's probably no VR plans for macOS ever. Apple's version of VR, whatever it ends up being, will definitely not be a tethered experience. It's most likely we'll see a specific OS variant like with watchOS when/if they do it.

mrex• 4 years ago

I would be beyond shocked if Apple's VR set doesn't include a Thunderbolt port.

koujaku14• 4 years ago

Cross-platform has always been a pretty important element of StereoKit! Right now there's not really a Mac headset, so there's no rush to dive in just yet. When something does come out, we'll evaluate it and fit it somewhere on the priority list :)

yessirwhatever• 4 years ago

What’s the current landscape of mixed reality devices?

Stereokit – Open-source mixed reality library by Microsoft

Related Stories