Ray Tracing with Metal [video]

https://developer.apple.com/videos/play/wwdc2019/613/

by pjmlp • 7 years ago

139 182 7 years ago

developer.apple.com

Apple doesn't sell any products with NVidia RTX hardware.

Yet still they feel the need to re-invent the wheel and write an API for it. I guess they really want to maintain that competitive moat.

sleavey• 7 years ago

Is this not something to do with the new Mac Pro?

robert_foss3• 7 years ago

The new Mac Pro only ships with AMD GPUs which do not yet support raytracing.

kraig9112• 7 years ago

Mojave as well does not have signed drivers for Nvidia.

Entalpi2• 7 years ago

Apple does though design its own GPUs. So it is not a far stretched idea that they might someday ship a GPU with dedicated RT hardware.

leoc3• 7 years ago

They're also likely to kiss and make up with nVidia again at some point, too.

kraig9112• 7 years ago

The iOS gaming platform is their biggest though? Think mobile gaming.

Chazprime2• 7 years ago

ATI is also bringing out hardware with dedicated RT cores as well, so it's likely that the hardware is coming to Macs shortly, just as next-generation game consoles will.

Metal is Apple's realtime API, and seeing as DirectX doesn't run on Macs, they'll need an API for raytracing in order to take advantage of ray tracing hardware, just as Vulkan did.

leoc3• 7 years ago

And if Apple doesn't provide a Metal RTRT API for third-party devs to target now, then whenever the hardware acceleration does show up third-party apps won't benefit from it right away. Apple will want the new hardware to bring immediate, large performance boosts to existing apps, not months and years of delay and hesitation from ISVs. Besides, they couldn't do any software RTRT themselves without writing a library for it anyway.

mcphage2• 7 years ago

Ray Tracing has been around for years. Heck, POV-Ray has been a great open source app for decades, even. It’s not some NVidia secret sauce.

zamalek2• 7 years ago

POV-Ray

Is not realtime. You could find an article on raytracing and code something in a weekend. Granted, POV-Ray is vastly more complex than a toy raytracer, but it is not realtime. It takes minutes to hours to render a single frame.

"Computers have been around for years, punch-card mainframes have been great at universities for decades even."

NVidia are, currently, the only vendor who have dedicated silicon for it (AMD have demonstrated DXRT working with existing silicon).

They do a actually have a secret sauce.

DerDangDerDang• 7 years ago

RTX is a marketing term, not a new technology.

pixelpoet• 7 years ago

Wrong, it refers to new dedicated logic on their GPUs for doing BVH traversal and triangle intersection, much faster than if you use the more general purpose shader units.

The same way GPUs had dedicated logic for rasterisation to make that super fast (hierarchical z-buffering, interpolation and blending), now they have some for ray tracing.

mtgx• 7 years ago

It's very disappointing that Apple hasn't adopted Vulkan. It's not like they are completely against standards either. I believe they even joined the FIDO Alliance recently.

With Apple adopting Vulkan, it may actually stand a chance against DirectX and the majority of game developers might start developing for Vulkan first. And Apple would benefit greatly from that, just like every other Vulkan supporter would.

tomxor7• 7 years ago

Yes, I find it difficult to imagine developers that are enthusiastic about "metal", I mean who wants another platform specific API for graphics cards. If Apple really think we need something other than "vulkan" then they should push to make "metal" a competing open standard- But I doubt they think that, and It's quite easy to imagine ulterior motives, M$ had exactly the same motives.

frou_dh3• 7 years ago

If Apple really think we need something other than "vulkan" then they should push to make "metal" a competing open standard

Metal predates Vulkan by years. They were already using Metal in shipping products before Vulkan existed.

pjmlp21• 7 years ago

All of us that rather use productive SDKs, with higher level languages, instead of a C API that only knows about drawing pixels.

The same devs that rather use DirectX, libGCM, libGNM, GX, GX2, NVN.

flohofwoe3• 7 years ago

I'm quite enthusiastic about Metal, and I'm not even much of an Apple fan. Compared to Vulkan or even OpenGL, Metal's API design is much more elegant and developer-friendly. It can be used just fine without a "sanity layer", which can't be said about Vulkan.

auggierose• 7 years ago

Given they make their own hardware, it makes sense to push their own standard, especially with the kind of market penetration they have. Now if it would only be possible to run Metal in the cloud ...

mbroncano• 7 years ago

Although not an ideal solution, MoltenVK [1] does a great job at bridging that gap

[1] https://github.com/KhronosGroup/MoltenVK

vondur• 7 years ago

Does any OS natively support Vulcan? Or do you get the drivers from a third party?

Avamander2• 7 years ago

Pretty sure AMD on Linux can do Vulkan with just second-party stuff.

syjer• 7 years ago

If you consider that window, normally use third party drivers (by amd, intel, nvidia) which bundle by default vulkan.

Linux distributions and the more recent android versions?

pjmlp21• 7 years ago

Only Android, and on Nintendo Swift as 2nd tier API (1st tier is NVN).

Even on Windows, you need from get the drivers from OEMs, and they only work in old Win32 desktop mode, not on store/UWP.

It remains to be seen if Windows sandbox will allow for anything else other than DirectX.

shmerl22• 7 years ago

Pretty much every OS, except heavy lock-in oriented ones, where OS is controlled by some entity which prevents providing graphics drivers. In such case it's up to that entity to produce them, and in most cases their lock-in idea is against it.

klum• 7 years ago

As someone pointed out, it probably has to do with controlling the stack all the way from hardware to userspace. Increasingly, Apples UIs run on top of Metal and I get the impression they want complete control, not just "good enough". Which makes sense, it's what makes Apple worth it for many people. I totally get the inconvenience from a developers (or gamers) perspective, though.

leoc3• 7 years ago

FWIW there have been one or two (apparent) Apple employees on here who seemed sincerely convinced that Vulkan was a mess compared to Metal. I have no idea how true that is or how much difference it actually makes, but there it is.

pjmlp21• 7 years ago

Metal, nice OOP based tooling with several frameworks to support handling textures, fonts, materials, GPU live shader debugging, code completion on Xcode with static analysis, C++14 based shaders.

Vulkan, plain old C style API, full of extensions just like OpenGL, leading to several code paths, just the bare bones pixel drawing, go fish for random libraries for anything else, try your luck with RenderDoc or GPU specific debuggers, enjoy Notepad++ + UNIX shell style tooling support.

daenz• 7 years ago

seemed sincerely convinced that Vulkan was a mess compared to Metal

Oh god I hope not. I come from the API mess that was OpenGL over 3 major versions, and I was hoping all the lessons learned from that were applied to Vulkan.

whywhywhywhy• 7 years ago

sorry at this point we need to stop wishing Apple would adopt standards or wishing Apple would stop reinventing the wheel in their own bespoke stacks and just support what we believe in elsewhere.

ksec• 7 years ago

Vulkan is on Android, Direct X is on Windows, Metal is on macOS and iOS.

I don't see anything wrong with it. Just like Windows did't adopt Vulkan. And that is assuming Vulkan is good in the first place.

And most of the gaming development are now done with Middlewares like Unity and Unreal. The way I see it, more and more people will develop with Middlewares in mind, those API won't be touched by a shrinking amount of developers.

leddt• 7 years ago

Vulkan runs on Windows.

morio• 7 years ago

Why would Apple care about how DirectX is doing? They support 'Standards' and 'Open Source' to the extent that it benefits them and that's it.

Vulkan is not designed to be used by game developers directly but rather as a basis for games engines like Unreal or Unity. So I don't see how game developers would benefit from its widespread support as the majority of game developers will never have to deal with it directly.

That said Vulkan is an unmitigated disaster, though less so than classic OpenGL. Any API which fiddles with void* in 2019 should be put right back in a box. Security is mentioned exactly twice in the entire spec. Type-safety is mentioned twice also. And that is in a spec which is 900 pages long and incredibly complex.

johncolanduoni4• 7 years ago

Vulkan is not designed to be used by game developers directly but rather as a basis for games engines like Unreal or Unity. So I don't see how game developers would benefit from its widespread support as the majority of game developers will never have to deal with it directly.

Vulkan has significant benefits even if you're not writing Unreal or Unity. One of the biggest is that it removes a lot of the heuristics and guesswork that OpenGL drivers do (e.g. with regards to when to move things in and out of GPU memory) that cause a lot of the bugs and incompatibilities on different IHVs/cards/platforms. Its validation layers also leave the halfhearted, vendor specific debugging extensions in the dust.

That said Vulkan is an unmitigated disaster, though less so than classic OpenGL. Any API which fiddles with void* in 2019 should be put right back in a box. Security is mentioned exactly twice in the entire spec. Type-safety is mentioned twice also. And that is in a spec which is 900 pages long and incredibly complex.

Metal overtly has statically checked memory safety via ARC, but the reality is that the IHV libraries will segfault in response to semantic mistakes, just like Vulkan. A big difference is that Vulkan has cross-IHV validation layers that catch most of these mistakes during development, and they're only getting more comprehensive. It's totally trivial to write a wrapper over Vulkan that fixes the type/memory safety of the C interface; it's not trivial to create a validation system of anywhere near the quality of Vulkan's for Metal.

banachtarski3• 7 years ago

That said Vulkan is an unmitigated disaster, though less so than classic OpenGL. Any API which fiddles with void* in 2019 should be put right back in a box. Security is mentioned exactly twice in the entire spec. Type-safety is mentioned twice also. And that is in a spec which is 900 pages long and incredibly complex.

OK are you a graphics professional? Have you written renderers in Vulkan? Do you have any idea of what you're talking about? What "void*" pointers are you referring to? Are you aware that type-safety is very much a design goal of the API? Are you aware that the spec ships with a formal memory model which is a godsend for graphics engineers that had to deal with implicit guarantees in other APIs for years? I'm not someone that authored the original spec, but reading your comment is somewhat rage-inducing given how off-base it is.

atq2119• 7 years ago

Pretty much all of the major game studios already have Vulkan implementations for their engines because of Stadia, Google's game streaming platform. Obviously we've yet to see how much staying power that will have.

Const-me3• 7 years ago

You can't adopt stuff which doesn't exist.

Apple was the first to release next-gen 3D API, Metal was released in 2014. Then came MS, DX12 was launched with Win10 in 2015. Vulkan was too late, initial release in 2016.

knolan2• 7 years ago

What about AMD’s Mantle?

vetinari2• 7 years ago

Metal 1.0 was ultra-limited; it was more MVP than anything else.

It was done exactly for the Apple to be "first". The initial Vulkan release was more comprehensive than Metal 2 in 2017.

Someone2• 7 years ago

Ignoring both technical merit and politics, both of which I’m not qualified to judge on, one reason they didn’t adopt it may be that it didn’t exist when they switched away from OpenGL.

https://en.wikipedia.org/wiki/Metal_(API):

”Metal has been available since June 2, 2014 on iOS devices powered by Apple A7 or later”

https://en.wikipedia.org/wiki/Vulkan_(API):

”The Khronos Group began a project to create a next generation graphics API in July 2014”

Of course, Apple could have changed direction later on, but they must have had a large investment in Metal by the time it became clear Vulkan would become a long term thing.

bangonkeyboard3• 7 years ago

Johan Andersson (then of DICE) was a presenter at Metal's unveiling. Andersson was one of the prime movers for Vulkan and Mantle before it, dating to 2013. Apple was very aware of Mantle/Vulkan while designing Metal, just as Apple was very familiar with CUDA when it released its own ill-fated OpenCL alternative.

shmerl22• 7 years ago

That's not really a valid reason, since not only Apple knew all along where it was heading (from Mantle times, to a common open low level API), nothing stopped them from supporting Vulkan once it became official. With Apple's resources, it's peanuts for them. So it looks like a completely political, nasty lock-in stance.

legohead• 7 years ago

Apple, please add a time multiplier to your video player. There's no need to watch a demonstration like this at 1x time.

DanBC• 7 years ago

Doesn't right click > play speed work?

fishtacos• 7 years ago

Not on Chrome on Windows - could be it falls back to a different player that allows that on other platforms/browsers.

tuacker• 7 years ago

Apple is using just a bog standard <video>-tag, so this should be a general plea to browser vendors (including Apple/Safari) to add that functionality.

olliej3• 7 years ago

I vaguely recall the video element already having this functionality - I remember an engineer working on it making the playback rate be negative. The performance was not spectacular :)

dzhiurgis• 7 years ago

There's bookmarklets for this

https://github.com/sjurgis/Bookmarklets/blob/master/youtube-...

pascoej• 7 years ago

document.getElementById("video").playbackRate = 1.5

Abishek_Muthian3• 7 years ago

One application which has impressed me with metal implementation even on low end HW (intel HD 620) is iterm2[1] terminal emulator.

IMO, it is a perfect example of how normal screen drawing if made using HW accelerated API could make a huge difference. This is especially true with low end hardware e.g. if Raspberry Pi (Broadcom/Videocore) had implicit support for OpenGL from the start by default like Apple has done with Metal, there could have been a decent desktop experience which is not possible with current SW rendering on low power CPU.

GPU rendering talk is getting lost with discussions about High end graphics cards or GPGPU computation, but I think Metal has brought back the focus to general screen drawing and judging from how developers like George Nachman (iTerm2) was able to take advantage of it; Apple seems to have done good with it.

[1] https://gitlab.com/gnachman/iterm2/wikis/Metal-Renderer

Teknoman117• 7 years ago

If you've ever played with Wayland on a raspberry pi, it's an amazing experience. It doesn't feel slow anymore. Now, apps still run slowly if they are doing anything particularly compute intensive (chromium), but it's very "responsive".

Abishek_Muthian3• 7 years ago

Yes, these are my experience. Please suggest if I had missed something which could provide a better experience.

My configuration to minimize variables which diminish Pi desktop experience - USB SSD boot, Zswap, Btrfs on Arch Linux 64bit (for better support for the latter 2).

1. Enabling VC4 driver at boot, enables OpenGL support to Pi.

2. For Wayland compositor, I used Enlightenment (Gnome, GDM didn't launch). I enabled HW rendering via OpenGL for all default rendering. Basic windowing experience is fast, as expected from HW rendering. Few default applications like terminal (Terminology) if used carefully(not resizing windows, not dragging etc.) works. But any other applications which use OpenGL like screenshot (or) applications which use XWayland chromium, firefox failed to launch. So, my conclusion is that this setup works only for basic windowing i.e. file transfers and careful use of console.

Perhaps Wayland via SW rendering (likely the one mentioned by you) could provide a crash free experience, but that defeats the purpose of getting general screen drawing HW accelerated (to feel responsive)?

gnachman• 7 years ago

Thanks!

Abishek_Muthian3• 7 years ago

Thank you.

astrange2• 7 years ago

Even without Metal, all drawing on macOS/iOS is "hardware-accelerated", especially if based on CALayers. Performance comes from focusing on performance no matter what layer you use.

shmerl22• 7 years ago

What's the story with Apple still refusing to support Vulkan natively?

AHTERIX5000• 7 years ago

They don't need to. Also Metal isn't really 1:1 comparable to Vulkan in terms of abstraction level.

shmerl22• 7 years ago

Same way MS didn't need to support common HTML/JavaScript since they had ActiveX?

If Apple would have acted properly, instead of their usual "eat our lock-in" mentality, they could provide Vulkan support, and then build higher Metal-style abstractions on top of it. But no, it's Apple for you. Eat Apple only Metal or get lost.

SomeOldThrow• 7 years ago

They don’t need to do anything. It would be a more attractive platform to develop for with a modern cross platform graphics api.

kjeetgill• 7 years ago

I don't know how closely you're following this, but MoltenVK[0] which was formerly a proprietary option for Vulcan on Metal is open sourced now.

0: https://github.com/KhronosGroup/MoltenVK/blob/master/README....

shmerl22• 7 years ago

Sure, it's a workaround for the dumb Apple's refusal to support it natively. It's great of course to break the lock-in, but it only highlights the problem. Translation is always suboptimal to the native option.

Impossible3• 7 years ago

It's a shame that Hackernews posts about graphics API features are so often derailed by discussions about Vulkan adoption. I'd be fine with the discussion if it was in context (a critique of Metal or even Vulkan) but threads about new Metal API features and even threads about WebGL and WebGPU are increasingly filled with the same fly-by responses about Apple's lack of support for Vulkan from people that have never used the API and are not graphics engineers. It would be much more interesting to have discussions about the merits of different approaches to raytracing in Metal vs. DXR vs. Vulkan than read the same tired half baked concerns about the existence of Metal, although it's a good chance for me to give Pjmpl more karma I guess. No one complains that Apple doesn't implement the entire Android API for iOS, why is it different with Metal?

Edit: The talk isn't even about new API features or raytracing hardware. The only reason it's "with Metal" is because it's a WWDC talk, it could be called "implementing GPU raytracing with compute shaders" and apply to Vulkan, DirectX, OpenGL or any other modern graphics API. Another good talk along these lines is this one by Matt Swobda (https://youtu.be/ZrFkKrw1uGI).

Reelin3• 7 years ago

why is it different with Metal?

Because the GPU ecosystem has historically been fractured. Vulkan looks like it has the potential to finally end that, but Apple has a walled garden and doesn't seem to be on board.

But I agree, such things aren't all that relevant to the article being discussed here.

Impossible3• 7 years ago

Have they? OpenGL was really the only option on most platforms for years. Even Windows has always supported OpenGL even if driver support from various vendors wasn't always good and official Microsoft support lapsed. Consoles have always used propiertary APIs, and before someone brings up PS3 and PS4 OpenGL implementations they are wrappers over low level APIs, so if they count then MacOS and iOS do support Vulkan via MoltenVK (there is even an official SDK on LunarG). If anything we've seen more fragmentation in the last 5-6 years due to the brief introduction of Mantle, followed by Metal and Vulkan. Some of this was necessary because OpenGL and DirectX development had stagnanted.

pjmlp21• 7 years ago

To discuss 3D APIs with their intrinsic value, instead of FOSS religion about them, we are better off visiting Gamasutra, Flipcode (sadly long time dead), Gamedev, IGDA forums, IGF,Game Connection, Making Games, Develop among many others in the game development scene.

I have long time ago learned the hard way, how the values between both communities are completely different.

Const-me3• 7 years ago

Programmers have a tendency to over-generalize stuff.

The idea that we need one true programming language, or one true GPU API, or one true text encoding, and use it everywhere, looks good on the surface. It’s when one digs deep enough into technical details, it became clear why people pick less portable technologies. It’s always about tradeoffs.

For GPU API, I think #1 reason is vendors support.

On PCs, Windows is _the_ target platform for majority of game developers. The only GPU API with first-party support is DirectX. GPU vendors do descent job supporting GL and VK there (except maybe Intel), but first-party support is still better. Major parts of D3D work in the kernel, dxgkrnl.sys does quite a lot of things.

On Android and many embedded Linuxes, it’s similar with GLES. Vulkan support is improving fast, I hope in a few years VK will replace GLES there, but so far there’re many older devices.

Ray Tracing with Metal [video]

Related Stories