Feasibility of low-level GPU access on the Web

http://kvark.github.io/web/gpu/2018/02/10/low-level-gpu-web.html

by kvark • 8 years ago

128 128 8 years ago

kvark.github.io

jcoffland2• 8 years ago

For the Folding@home project we would love to have GPU compute access widely available from the browser.

kvark2• 8 years ago

If you just care about GPU compute and not much about the shape of the API or it's overhead, then something like OpenGL ES 3.1 Compute Shaders or WebCL would be much easier to reach than WebGPU, technically.

johndough2• 8 years ago

Mozilla won't support WebCL in favor of compute shaders, but that announcement has been over 4 years ago and there still is no general compute in browsers: https://bugzilla.mozilla.org/show_bug.cgi?id=664147

gambiting• 8 years ago

How many people do you think are actually interested in running the client in browser? I have Folding@Home on about 5-6 machines and I can't imagine a situation where I could run a web client but not a full client.

RandomInteger4• 8 years ago

Chromebooks?

bastawhiz• 8 years ago

It's not an incredibly common use case, but many computers exist simply to show a dashboard or visualization. It would be excellent to say "drop this script tag on your page" and you instantly turn that Mac Mini into less of a waste.

sitkack7• 8 years ago

It isn't for you. It is for the person who can open a tab, but doesn't want to run a full client. Installation, sandboxing, laziness, etc.

A university or corp could push a tab open script to thousands of desk tops, while they wouldn't push Folding@Home client.

jcoffland2• 8 years ago

A fair number of people are already running the Chrome NaCl client in their browsers but it is CPU only.

http://folding.stanford.edu/nacl

bobajeff6• 8 years ago

I have this idea that if they split off WebAssembly, WebGL and WebAudio into a external format for applications it would be easier to implement than a full browser.

rhelmer3• 8 years ago

You could probably do this now as a standalone Node app, if you want more explicit control over which APIs are available and where code can be run from.

The browser is indeed quite complex (and not just because of the massive historical baggage), but its job is to give the user control while safely downloading untrusted code and running it locally.

So, if your goal is to just have a simple standalone app I think you could stay largely compatible with APIs available in the browser environment.

bobajeff6• 8 years ago

Easier implementation is the goal. There are currently only four companies working on a web implementation.

Also, I think users would have more control if documents didn't automatically gain the same privileges as applications.

leoc2• 8 years ago

You could even implement some kind of … HTML document-reader webapp on top of that. [thinking face emoji] https://news.ycombinator.com/item?id=2131870

Sir_Cmpwn• 8 years ago

http://trevorlinton.github.io/

floatboth2• 8 years ago

I think I might have seen a canvas-only embedded "browser" without the DOM somewhere on github already…

avaer2• 8 years ago

I've gotten pretty far doing exactly this (only-those-apis browser in Node): https://github.com/modulesio/exokit

Not released yet, but there's basic Windows, macOS, Linux, Android and iOS support. Contributions welcome!

bobajeff6• 8 years ago

Your ideas is actually really cool (very tech noir).

But that's not what my idea is. My idea isn't really a browser. It's just a separate format/mimetype (.app or .game) for a app runtime.

Small projects could implement it. We can have it embeddable as a object in browsers. Clients other than major web browsers (Gopher, Dillo) could embed it. Embedded devices (Roku) could include support for it. It could even be used in a physical media like SDcards or DVDs.

imhoguy• 8 years ago

Flash-back!

sitkack7• 8 years ago

Couldn't many of the GPU operation be accessible via functions over typed arrays? How does this relate to using SIMD from JS?

baybal22• 8 years ago

It has same downside as WASM. Running precompiled bytecode is a bad for security as it always been.

It was no more than a year when a remotely exploitable WASM hole was exposed (derivatives of Spectre and co.) Knowledgeable people told that ISA level hole that can be exploited remotely over the web will be "a one minute global IT disaster" if somebody would resort to propagating it through a big adnet or paid traffic scheme.

As for WebGL as it is now, there were numerous sites on my memory that froze/crashed/rebooted both Linux and Windows systems, which means that the prime suspect there was a buggy shader as it is the only thing resembling raw instructions that can be passed to gpu through webgl.

Felz• 8 years ago

Weren't Meltdown/Spectre exploitable via Javascript?

dralley• 8 years ago

Yes

baybal22• 8 years ago

Actually, the POC required both sharedbuffer object and ASM js (actually an even worse thing than a bytecode)

jandrese• 8 years ago

You are being downvoted, but it is important to remember that graphics APIs were not developed with security as a first class requirement. They tend to be large, arcane, and often interfacing with large binary blob drivers on the system. Their threat surface is enormous. IMHO it is just a matter of time until exploits for WebGL and the like start showing up regularly.

Retr0spectrum• 8 years ago

Maybe GPUs need to be fundamentally redesigned to allow multi-user access with enforceable security boundaries. I'm sure this would be highly non-trivial to implement, but it would be great for other scenarios like shared access to GPU compute resources in "the cloud".

kvark2• 8 years ago

This is called GPU virtualization, and there have been experiments ("Sugar" [1]) of using it for WebGL. We looked into it, consider it promising but not urgent. It addresses some of the security concerns at the cost of performance (and implementation complexity), but the portability issues are unchanged.

[1] http://newport.eecs.uci.edu/~amowli/hpcfactory/publication/a...

monocasa2• 8 years ago

That has existed for over a decade. In fact that's sort of the point of Vulkan, et al.; with all user code being behind an MMU context, it's safe to provide a more console like 'bare metal' API. There are bugs, but the infrastructure is all there. The end goal is that you can only crash your own process.

The issue with this in the browser is that the API isn't part of the JavaScript/WebASM, and just exposing it in the same way will allow you to subvert the sandboxing of the VM>

happyguy43• 8 years ago

How does this interact with WASM?

Animats2• 8 years ago

What problem does this solve for the end user?

Few sites even use WebGL for anything interesting.

na85• 8 years ago

Most of these fads seem to be driven not by what's good for the user but what's good for the advertiser and the developer.

Low level GPU access will just make it easier for them to run montero miners on my browser.

Animats2• 8 years ago

There's an ICO for that.[1] Really. They want to offer browser based virtual reality which mines tokens in the background. The initial token sale is in progress now. The virtual reality system appears to be total vaporware.

[1] https://55cancri.io/

pitaj• 8 years ago

Monero isn't efficiently mined on GPUs (it works better on CPUs which is why it's so popular for browser mining already), but other cryptos with equihash algorithms, sure.

ghostwreck• 8 years ago

The future of AR/VR exists in 3D worlds. If the web wants to be part of that future, developers will need access powerful 3D APIs. Otherwise, the web will miss another tech wave like it missed mobile (as native mobile was the clear winner over web mobile).

digi_owl2• 8 years ago

So we are back at VRML?

randomerr• 8 years ago

I see it more as way to use the GPU to do calculations in JavaScript similar to the speed C/C++ does for crypto-currencies. Yes you can create a Native Application (NaCl) but that is only going to work in Chrome/Chromium browsers and it's compatible variants. Electron and NodeWebkit/NWJS based projects are the most common. But NaCl still doesn't give you direct GPU access because of the sandboxing.

fabian2k• 8 years ago

I've used WebGL to visualize scientific data. It's nothing special, but the size of the data was large enough that anything like SVG or even Canvas were just slow. There is a lot of interesting stuff you can do even just in 2D in WebGL, because you're closer to the hardware.

What's annoying is that you don't get double precision or any of the compute stuff on WebGL currently. The interesting stuff isn't just graphics, but also doing computations directly in the browser.

KitDuncan• 8 years ago

Do you have any resources on how you did that? I am developing a web app and I have to visualize a ton of datapoints in a bunch of charts and all the charting libraries I've tried are way too slow. I've been looking for something that could utilize WebGL to speed things up, but haven't found anything so far.

c-smile3• 8 years ago

This situation reminds me days when we, at W3C HTML5 WG, were trying to sneak SQL specification into HTML5 "umbrella". And one particular flavor of it - the SQLite's SQL as it was defined at the moment of writing. Haven't got through for many good reasons as we know.

As of GPU exposure to the Web ...

It makes sense only when we will have stable and unified GPU abstraction. As for now DirectX.12, Vulcan and Metal are close but different.

Like the WebGL that is (more or less unified) OpenGL. But even that looks too foreign to HTML/CSS/script - immediate mode rendering in architecturally retained display model of web documents.

And conceptually: HTML5 umbrella is large but not infinite. 3D rendering is too far from HTML "endless flat text tape" model.

I remember those <applet> days when browser was used as a component delivery platform for stuff that does not fit into DOM and structured yet styled text. That was conceptually the right way "to grasp the immensity".

These days, with WebAssembly, we have another incarnation of the <applet> idea and I think GPU functionality belongs to it rather than to HTML. GLSL to be expressed in WebAssembly bytecode terms but not in JS.

Web standards have to be backward compatible and I doubt that current still ugly GPU paradigms will survive on the long run. Like tomorrow someone will come with practical voxel based system instead of current abstract vector ones, what will we do?

revelation2• 8 years ago

This is a bit of a rambling mess, but to just make one point: the winner has been chosen. Android is implementing Vulkan, and so Vulkan it will be.

dleslie2• 8 years ago

It will be Vulcan and Dx12. Microsoft is nothing if not a stubborn 800 lbs gorilla.

orbitur• 8 years ago

If Vulkan is the future, I wonder how mobile devs will make money if they can't target iOS.

Rusky• 8 years ago

Like tomorrow someone will come with practical voxel based system instead of current abstract vector ones, what will we do?

For one thing, any new approach will still have to solve the same problems around data transfer and formats that make up a lot of current APIs.

And for another, no matter how good this hypothetical new approach is, the old one will still be around to handle existing content and workflows forever anyway.

colordrops• 8 years ago

HTML and CSS are slowly losing ascendancy as the main render surface for web apps. The canvas provides a standard pixel buffer for both JS and compiled applications, making HTML's limitations no longer a restriction.

lr4444lr• 8 years ago

From your lips to God's ears.

stu_k• 8 years ago

Except that rendering everything to canvas is not accessible to people with certain disabilities.

Also mentioned recently here https://news.ycombinator.com/item?id=16347216

c-smile3• 8 years ago

<canvas> by its specification mandates pixel buffer to be placed in memory with per pixel access. This reduces GPU acceleration options to the ground.

jacquesm• 8 years ago

The whole point of the web was not to have people write directly to bitmaps, to leave the presentation up to the rendering platform.

In the quest for eye candy we're losing what made the web great to begin with.

pdkl95• 8 years ago

compiled applications

A lot of HN comments dismissed my prediction[1] that WebAssembly will bring opaque "compiled applications" that treat the canvas as a "standard pixel buffer", allowing adblocking CSS and request filters to be bypassed.

A lot of people seem to be focusing on the benefits of new technological changes ("no longer a restriction"), when they should first be concerned with the potential risks that change will create.

[1] https://news.ycombinator.com/item?id=10211050

erikpukinskis• 8 years ago

I’ve actually noticed a return recently to simple “HTML native” UI design on the web. Not as many people trying to emulate an “OS native” experience by adding a bunch of extra DOM elements to render borders and effects and such.

ZenoArrow• 8 years ago

"This situation reminds me days when we, at W3C HTML5 WG, were trying to sneak SQL specification into HTML5 "umbrella". And one particular flavor of it - the SQLite's SQL as it was defined at the moment of writing. Haven't got through for many good reasons as we know."

Can you elaborate on the arguments against a dialect of SQL being available for use with local storage? I can only think of arguments in favour of it. Would be good to understand the grounds on which the idea was dropped.

thristian• 8 years ago

The lesson of the Web is that when you expose some interface for people to use, they will start depending on documented features, undocumented features, and downright bugs of the first popular implementation. Second and subsequent implementations will need to spend a bunch of time reverse-engineering those bugs so they can be documented and reliably implemented in future so that existing websites keep working.

The only alternative is to make sure all new interfaces have multiple, popular implementations simultaneously, so that authors can't afford to take advantage of bugs in one implementation. This is the sort of activity you see in WHATWG these days.

The problem with "a SQL dialect" being available for use on the Web is that every browser intended to use SQLite as the backend. Nobody wanted to invest the time and effort to write a second, compatible, equally-reliable implementation; nobody wanted to exhaustively research and document bugs and flaws in that specific version of SQLite; nobody wanted to commit to back-porting security fixes from later versions, or forward-porting the required bugs to later versions.

And since nobody wanted to do the work to make it possible, using SQL for local storage remains impossible.

kevingadd2• 8 years ago

The SQL situation is a bummer. It made sense to avoid a monoculture, but indexedDB is a near-useless wreck due to its terrible performance and it has no meaningful querying primitives at all so you get to implement them yourselves and the performance is even worse.

Wish we could've just had two SQL implementations. There are other ones out there - Microsoft ships JET and an embeddable version of MSSQL, someone could've embedded postgres or something. As-is people who care about performance are just going to compile SQLite down to wasm/asm.js and run it in a worker.

seanmcdirmid2• 8 years ago

Canvas is immediate mode rendered and already breaks through the DOM. There are also popular movements to make the DOM appear more as an immediate mode abstraction (React). In contrast, most retained mode 3D abstractions are seen as pretty bad.

pjmlp12• 8 years ago

The majority of game engines scene graphs are retained mode 3D abstractions.

Everyone thinks that they can do better with an immediate mode API, then we start building some kind of data structure to track down what needs to be drawn and when, with the result being similar to the traditional joke of half-implemented Lisp or ORMs, but applied to retained mode rendering.

Sure, some experts manage to get it right, and those are the ones that get to write AAA game engines, but many don't.

fulafel3• 8 years ago

WebGL 2 already gives you pretty good GPU gfx & compute. The use is limited by implementation quality in browsers. Let's give it time.

Marat_Dukhan2• 8 years ago

WebGL 2 is based on OpenGL ES 3.0, it doesn't give you compute. Compute shaders were added in OpenGL ES 3.1

blarg1• 8 years ago

khronos said they were working on compute for webgl2, who knows when it will be ready though.

fulafel3• 8 years ago

The OpenGL compute shader feature is not needed for doing GPU compute. It's just another type of shader that is not connected to other GL rendering that may be happening at the same time. People have been going GPGPU with the traditional types shaders for a long time. And WebGL 2.0 is a huge upgrade from 1.0 from a GPGPU point of view.

Also, a compute shader extension may be coming[1] for WebGL 2.0, it's mentioned in some meeting slides at least: https://www.khronos.org/assets/uploads/developers/library/20...

swix• 8 years ago

Canvas and webgl is lovely in the way it quickly lets you get something on the screen... Pixel pushing on canvas is, easy and accessible.

It makes me wonder if anyone has created some sort of port to a standalone app with no browser involved where you could use javascript/canvas-api/webgl to draw pixels on a canvas-like surface... without the fatness of the browser. Just spawning some window, that would be a lovely scripting/game-dev environment, maybe with some sdl-bindings or whatever. Anyway just rambling, does such a project exist? Anyone knows? :)

tambourine_man• 8 years ago

Canvas was introduced and inspired by Apple's drawing tech, so you might want to check that.

icebraining• 8 years ago

I suggest https://love2d.org/

sitkack7• 8 years ago

I think you meant this, love2d running in the browser.

http://tannerrogalsky.com/mari0/

haxiomic• 8 years ago

I built exactly that a few years ago: V8 with WebGL bindings that passed through to OpenGL ES. It was to enable WebGL experiences on the GearVR, where a full browser wouldn't cut it.

I've got sample using an OS X window but it should be easy enough to do the same for Windows

It's been sitting in a private repo since then but I can give you access if you'd like

That being said, I think a better alternative for a thin graphics scripting environment would be haxe with Lime or snowkit (which provides OpenGL, SDL and windowing). I've used these in the past and loved working then

visarga• 8 years ago

Hmm... I wouldn't want a random webpage to start mining bitcoin on my computer. How would we protect against that?

ramses0• 8 years ago

Net Neutrality, right? It doesn't matter what the page does, it only matters that you are allowed to visit any page on "The Inter-Web" that you like, right?

Safari has "tab paused/reloaded due to high power consumption" (good), Chrome has "auto-mute tabs" extension (which I have turned on), I have "tab suspender" extension installed.

Basically, I'm trying to be a responsible consumer:

- this tab would like access to your hard drive files

- this tab would like access to your video camera

- this tab would like access to your microphone

- this tab would like to play sounds

- this tab would like to download more than 5mb of data

- this tab would like animation/movement

- this tab would like to use your CPU a lot

- this tab appears to be using a lot of your battery

- this tab would like to use your GPU (at all)

- this tab would like to maintain state > 24hr

the_internet.js is actually potentially really hostile (suck 9999mb at full speed, ddos@1.2.3.4, while(1){alert(1)}, mine_bitcoin( $hacker_wallet )), and I am much in favor of treating it as untrusted by default (low access, limited # of cpu cycles) until "trusted" (ie: android permissions swap: i give you the executable, you give me the permissions).

YouTube? Yes to whatever they ask.

ShadyWebsite.1234.some-random-domain.ru? You can d/l 300kb and can't do anything else (ie: web 1.0/no-script).

While each "tab" in a web-browser attempts to provide "safe" access to the computer resources, it is still not "permitted" access to computer resources. Whatever the browser defines as "safe" is 100% ok, which has to work equally well for WASM-unreal-tech-demo-castle as well as cnn.com.

I'd prefer cnn.com only had 300kb download, no external domains, no sound, no battery, no cpu, etc.

As I _trust_ cnn.com more (to the same level as youtube), I would then permit sound by default, permit large downloads, permit gpu, permit animation/video/etc.

z3t4• 8 years ago

i think the web browser is the modern terminal and it should make use of all capabilities of the device, including hardware accelerated rendering. i think the current webgl is too low level though and only needed if you want to make your own 3d engine. if it was a higher level 3d renderer it would be easier for browser wendors to make secure and optimize, and easier for developers to use.

rimliu• 8 years ago

And I start to think that web tech is becoming a fast food of software development.

matt_m3• 8 years ago

Well, as a small developer I'd love to see a higher-level, easier-to-use API (than Vulkan) that can be used as a modern, cross platform replacement for OpenGL. If it could be used both in a browser and standalone even better. If it was available outside of a browser (even as a Vulkan/Metal wrapper), I think it could become a no-brainer replacement for where OpenGL ES is used today.

I understand that engine developers can get some (small?) percent improvement from an ultra-low-level API that exposes more platform specific details. But they almost always support multiple APIs natively, and already use the low level ones where available. There may be a small performance benefit (over a higher level API) in the browser, but I don't think the browser is a likely target for ultra heavyweight apps / games, that usually are multi-GB downloads anyway, in the near future. And keep in mind that if smaller software sticks with WebGL because of API complexity, that might be a big performance loss for the user.

But a higher level API would have great benefits everywhere! Right now the only real cross-platform graphics API is OpenGL ES 3.0, which is becoming more and more obsolete, with little sign of the situation changing. No compute shaders, no AZDO, etc. Any move beyond that feature set now requires multiple APIs, shaders etc. And the easiest way for a small developer to get those features is still to skip Vulkan and to stick with GL ES 3.1+ on Win/Linux/Android and choose Metal on Mac/iOS. Of course on the web for those features there are no options at all.

I think if a new API was available that was easy to use and truly cross platform (including web), it would be the obvious first API to implement for all new graphics software. And this would be a much larger benefit than an unknown performance improvement that is accessible mostly to engine developers.

Vekz• 8 years ago

Not cross platform (WebGL only) but Regl: http://regl.party/ Is, In my opinion, the way GLSL GPU bindings should be done.

zanny• 8 years ago

If you are making a commercial product Vulkan + MoltenVK should be perfectly serviceable as a target. There are also free software attempts to make a wrapper.

At this point, so long as Apple is in your target market, there will never be an everywhere API because Apple does not want there to be one. The whole point of Metal is to make your life harder so developers currently writing for Apple first are less likely to port their software elsewhere.

matt_m3• 8 years ago

Yeah that seems like that might be the best option right now, and it doesn't look like it will be too expensive either. What I probably would like more is the reverse, a Metal wrapper for Vulkan, since it seems so much easier to get started with Metal. Too bad Metal is a Swift/Obj-C API so it's not straightforward how to make it cross platform.

I don't know how much bad faith I want to assume on Apple's part, since there are probably some legitimate technical reasons why they don't want to support a lower level API (and they came out with Metal first). Vulkan is such a pain to use anyway that it's probably mostly used by engine developers who generally don't have a problem supporting multiple APIs like Metal / DirectX etc.

But at the very least, it would be nice if they upgraded their OpenGL version, since they already support that and it's only them holding back some of the newer features.

matt_m3• 8 years ago

Maybe I could have explained this better since it's getting downvoted. Right now the two big modern graphics APIs are Metal (iOS / Mac) and Vulkan (Win / Linux / Android). These aren't the only ones, there are more for game consoles, UWP apps, etc, but they are arguably the most important ones.

Metal and Vulkan are not at the same level of abstraction. If you look up the code needed to draw a triangle on the screen (maybe the most basic graphics task), it is much, much more long and difficult to do it in Vulkan than Metal. Vulkan is a lower-level API. It gives engine developers more flexibility, at the cost of making basic things time-consuming and complicated.

OpenGL was the previous cross platform API. WebGL is almost identical to OpenGL. OpenGL still runs on Mac / iOS, but Apple has stopped supporting newer versions. The newer versions of OpenGL have closed the gap some with Vulkan and Metal (it won't catch up entirely, but it added some important features like compute shaders and it's easier to use it efficiently). OpenGL is still easier to use than Vulkan. The problem is the newer versions are not cross-platform, since Apple wants to focus on Metal.

Apple does not want to support Vulkan. Metal came out before Vulkan did, and it is a higher-level API. It's arguable if Apple should support it or not, but that's how it is. Microsoft also wants to focus on DirectX 12 (their API).

I was making the argument that a higher level API, Metal-style, would be a good base for the new web standard. Metal couldn't be used directly, at the very least it would have to be changed from Swift/Obj-C. But the idea of roughly basing it on Metal as mentioned in the article doesn't seem unreasonable, even Vulkan was based on a previous AMD technology called Mantle.

A low-level standard like Vulkan is hard to use directly, its adoption will depend mostly on people using frameworks / engines that use Vulkan. It's possible that due to its low-level nature there would be some performance advantage, although games using Metal also seem to get good performance on iOS. The disadvantage of Vulkan is that it is much harder to use than WebGL.

A fair amount of the WebGL content is not web specific. It is possible to write OpenGL content and compile it for desktop / mobile and the web. The most popular game engines, Unity and Unreal, both support compiling to the web and desktop from the same codebase.

In my view, there is no good replacement for OpenGL, now that new versions are not cross platform. Vulkan is much more work and does not work on Apple platforms, while Metal is easier but only works on Apple platforms. If they could provide a standard that is both easy and cross platform (by providing a C API library in addition to the web standard), it would provide the best of both worlds. The main downside is that it might leave some performance on the table compared to a low level API, and it would be yet another standard (which is why it would be important to provide a native library too, so developers have the choice of coding to only one API).

pjmlp12• 8 years ago

I would add to your remark that the very fact that OpenGL ES ever took off was because of Apple.

Before iOS, most mobile devices were having their own experiments with 3D APIs, Nokia N95 was the very first with OpenGL ES compatibile GPU, but it was thanks to iOS games that it ever took off.

Apple was also pursuing Quickdraw 3D before the NeXT acquisition, so not too keen on OpenGL anyway.

Their OpenGL's adoption was mostly survival related, now they are back on top, they can afford to dictate their 3D APIs just like all other console vendors.

guidedlight• 8 years ago

There is an upcoming Xbox game (Crackdown 3) that uses web-based GPU processing to share compute amongst players in a multiplayer game.

andrewmcwatters• 8 years ago

What does that even mean? It's an Xbox, they all literally have the same variations of hardware. There aren't 8000 different build specs, there's at most, what 8? How could compute sharing provide any possible enhancement in experience at all? If anything, it's eating frame budget and it shouldn't.

stordoff• 8 years ago

When it was first announced, there was talk of off-loading certain tasks to (presumably) Azure. That may be what OP is thinking of.

I'd also wonder if you could share the power of any Xbox One Xs connected to a multiplayer session, given the gap between an original Xbox One and the X is rather large (seems like it'd be far more trouble than it's worth though).

slavik812• 8 years ago

I don't understand why this is a priority when WebGL is still so rough. Maybe we wouldn't need a new API for performance if WebGL worked better. There seems to be lots of room for improvement. My WebGL programs were much slower and were harder to write than the native versions of the same programs.

We should also probably sort out the native low-level APIs before setting the standard for the web, because otherwise we're building on top of a big mess. Though, my impression is that the WebGPU initiative is basically just another battleground in that struggle. I don't have any faith that this is being done for the good of users. It's just strategic ground to capture.

stillkicking• 8 years ago

Because WebGL is an evolutionary dead end, for a variety of reasons. The initial idea was to track OpenGL ES, but that isn't really true anymore. Because of Windows, WebGL has to stick to a subset that can be easily translated to Direct3D. Because of GPU process sandboxing, anything that flows back from GPU to CPU is a huge synchronization problem. On top of that, mobile GPU drivers continue to suck badly, which means most of the extensions that were supposed to expand WebGL's scope into this decade are still out of reach, with 50% or less support in practice.

On the flipside, the native low-level APIs have all diverged. Vulkan, D3D12 and Metal each made different decisions, so it's pretty much inevitable that a 4th standard will have to be created to unify them. It will probably be higher level than any of the 3, and it will still be subject to strong sandboxing limitations.

Personally I think the big issue is that people stare themselves blind on the traditional graphics pipeline. Modern renderers have evolved past this, with various compute-driven and/or tiled approaches common place now. They're a nightmare to implement against current APIs, because a small change in strategy requires a rewrite of much of the orchestration code. The job of figuring out how to map your desired pipeline onto the hardware's capabilities should be the job of a compiler, but instead people still do it by hand. Plus, for GPU compute-driven graphics to be useful for interactive purposes beyond just looking pretty (i.e. actual direct manipulation), you absolutely need to be able to read back structured data efficiently. It's not just about RGB pixels.

There's an immense amount of potential locked inside, but the programming model is a decade or two out of date. Only AAA game companies and the vendors themselves have the resources to do novel work under these constraints. Everyone else has to throw together the scraps. Even the various attempts at LISPy GPU composition fall short, because they don't attempt to transcend the existing pipeline.

hasahmed• 8 years ago

I would like to hear more about this. I was quite surprised how difficult things were when I started dabbling in Opengl and I thought that there has to be a better way. I know that there are libraries that build on top of Opengl and the like, but then always its a sacrifice of the power that you could have. It seems weird to me that it is so difficult because conceptually it seems to me that the model could be closer to the CPU/Memory model that everyone is already familiar with. You just have some RAM and some processor(s) that are going to do some computations right? Although I guess what really makes it a mess is that there needs to be a connection between what the GPU and the CPU are doing. I don't know, I was a bit surprised by how difficult it was. Perhaps I just don't understand it well enough.

slavik812• 8 years ago

I agree, yet I still don't understand how WebGPU is going to get buy-in from Apple and Microsoft if previous attempts at defining cross-platform APIs could not. Any web API will be open and cross-platform. If WebGPU is well-designed, it could very well be adopted as the next OpenGL.

I would rather target WebGPU and write my program once rather than implement my logic three times in Metal, DirectX and Vulkan. But, Apple and Microsoft don't want me doing that, so why would they support WebGPU?

Feasibility of low-level GPU access on the Web

Related Stories