Render HTML via WebGL

https://github.com/PixelsCommander/HTML-GL

186 109 11 years ago

jeremiep• 11 years ago

It looks great, but I don't like the fact that I can't select the text anymore!

It can be selected at same bounding box DOM node have and the selection will not be highlighted at the moment, but we will fix that.

riskable• 11 years ago

If you can get text selection (and copy/paste) working please post how you did it. I wrote Gate One which is a web-based terminal emulator. Right now I have to go through great pains to ensure that text being rendered to the screen happens in a performant way while still allowing for copy & paste functionality. It also has some features that will highlight text that you select with your mouse and other text formatting stuff.

I've experimented with canvas in the past to see if it could improve the performance of, say, a terminal that updates very quickly but it ultimately ended up being just as slow as the regular DOM because of all the extra code that had to be running in order to keep the hidden buffer in sync with what was displayed to the user.

I'd be curious to know how the performance of WebGL compares to the (accelerated) canvas.

thecatspaw• 11 years ago

it looks blurry. does it support retina displays?

methyl2• 11 years ago

So we have browser in the browser, we have to go deeper...

PixelsCommander21• 11 years ago

It is basicaly not. Just converting HTML to texture with some tricky proxying. The reason is to be able to do that http://pixelscommander.com/polygon/htmlgl/demo/filters.html

MrDosu3• 11 years ago

What he means is that is exactly what the browser does already. It's just slower because it handles more of the standard.

It rasterizes the DOM.

outworlder• 11 years ago

Now we need a WebGL demoscene.

PixelsCommander21• 11 years ago

It is already there https://www.chromeexperiments.com/webgl , but it starts to be even more exciting if you think about effects which could be applied to web sites and HTML content... Bumb mapping, reflections, motion blur, elastic elements when dragging. Whole bunch of shaders https://www.shadertoy.com/browse

theandrewbailey• 11 years ago

There is. It's been around since WebGL was a thing.

http://www.pouet.net/prodlist.php?platform[]=JavaScript&page...

PixelsCommander21• 11 years ago

Cool, thx

MrDosu3• 11 years ago

Demoscene is not about flashy graphics. It's about doing something beautiful with code (this includes the way the code uses the machine to do things that go beyond, like mode 13h).

aout7• 11 years ago

Wait wait wait... Handling textures via CG Buffers is quite fast you are right and you can basically do whatever you want in terms of image manipulation using the shaders but you'll lose efficiency as soon as you introduce complex picking (text selection for example) and / or computations (aliasing etc...) between JS and your pseudo-DOM.

In the end the only difference you'll have compared to the real browser renderer is that you do less computations from CG(GL) to RAM (JS), hence the performance.

You can check this very old project of mine where we tried to link DOM and WebGL Nodes: https://github.com/aout/SAGE

edit: just saw the repo wasn't entirely up to date. Done now.

bsenftner3• 11 years ago

Could you speak a bit more on the issues you encountered?

aout7• 11 years ago

Well, your browser is already GPU-accelerated. What makes it slow is the process of making GPU-memory accessible to your JavaScript.

An example of that is drawing a DIV and applying a CSS 3D Transform to it. By doing so you'll modify it's display on the screen so you'll need to recompute it's actual bounding box. If you compute the BBox on the GPU and never access it anywhere else, everything is fine but as soon as you have to use it in JS you're screwed because you'll have to transfer data from your GPU to your RAM. Doing so at 60fps is very expensive, then you'll take a performance hit from JavaScript being usually slower than C/C++.

That's not the only problem you'll have, text rendering is probably the biggest one. As you might know, textures are rendered using memory friendly algorithms. To ensure maximum readability and quality you'll have to (1) use a lot of memory and / or (2) apply some advanced filtering so your texture doesn't lose quality. Those techniques are quite hard to implement if you're not a 3D programmer (just check what happens if you try to rotate text in CSS and select it) but more importantly you'll find yourself reinventing the wheel.

PixelsCommander21• 11 years ago

Nice, we may use it as a renderer for HTML GL. Shaders and fast rendering wirth it I believe. Try the demo in the bottom http://htmlgl.com/

aout7• 11 years ago

I tried the demo of course and I thought you did a great job. I'm not saying you should abandon the project because of performance issues you'll encounter but rather focus on simplicity with clear boundaries. Again, go for it! (but don't go too far)

danjayh• 11 years ago

I like the concept for graphics-heavy pages, but for any text-oriented page ... well, any website that didn't support highlighting (which I use to keep track of where I am on a page) and that didn't re-render when zooming (absolutely necessary for high-res displays, where a page might show up tiny and need to be re-scaled) would be a website that I didn't stay at for very long.

bsenftner3• 11 years ago

Really great work. I've had in the back of my mind that this was possible, but now you've gone and done it, and quite well!

I'm curious if there would be any conflict with usage of Three.js on the same site? Could they end up clobbering each other's textures? (I've not gotten into webgl's texture memory management, so that question is a stab in the dark.)

PixelsCommander21• 11 years ago

I would not expect any conflicts there since they are different canvases and textures uploaded just as files. You may ask mat@goodboydigital.com for more details.

punkbit• 11 years ago

Holy cow! This is a great idea : D the responsiveness is a bit sloppy but it's really good you guys have it working! haha really cool : D

PixelsCommander21• 11 years ago

Thanks! We are working on improving responsiveness a bit. Will be better.

designml2• 11 years ago

Ahh Nice! I was going to start working on something similar to this.

PixelsCommander21• 11 years ago

Come on! We can do much more together =) Still so many WOW things to implement...

designml2• 11 years ago

Haha yea, I am going to delve into your code and see if I can contribute. But nice work! Your site is probably getting hammered now with the traffic but I am going to come back and check it out.

danbruc• 11 years ago

This was at least kind of possible (not sure if it worked only for images or also other elements) with Internet Explorer 4 [1] in 1997, almost two decades ago. It was obviously non-standard.

[1] https://msdn.microsoft.com/en-us/library/ms532849.aspx

cordite• 11 years ago

Although non-standard, Microsoft sure did several novel things back in the day here. Then they decided not to do the actual standards correctly and held a lot of things back since nobody updated these things.

On the topic of Microsoft and the web, ActiveX gets a lot of bad rep (and for good reason), though it seems like they were trying something akin to web components back then. It worked better in actual applications than the web though.

kolodny• 11 years ago

fiddle: http://jsfiddle.net/409gxzm6/

bshimmin• 11 years ago

The text is blurred on my retina MacBook Pro. (And not selectable, as others have mentioned.)

PixelsCommander21• 11 years ago

Yes, there is an issue on that. https://github.com/PixelsCommander/HTML-GL/issues/11 I d like to fix it before going public but HTML5WEEKLY made announcement without notice, so everybody sees know that bug =( As for selection - it is selectable in it`s bounding box, but selection is not highlighted yet.

kluck• 11 years ago

Recreating existing desktop GUI frameworks one useless HTML/JS library at a time.

PixelsCommander21• 11 years ago

Come on http://pixelscommander.com/polygon/htmlgl/demo/filters.html

MrDosu3• 11 years ago

I don't get why people do this without looking at least once at all the existing open source html rendering implementations.

Also, css shaders.

PixelsCommander21• 11 years ago

What is the browser support for CSS shaders? =)

angersock3• 11 years ago

Massively underrated post, honestly.

I wonder what exactly the use of this is, beyond Compiz for web pages. What's next, jiggly divs? :P

moron4hire made a good point about letting devs pick the layout engine, but I am not sure we want to ignore all the nice features browsers give us here.

bsenftner3• 11 years ago

Would forms and input fields still work? Could one push their wysiwyg rich text editor into this? (I'm thinking game / VR content where an in-3D-world web browser on some in-3D-world device screen is fully operational.)

PixelsCommander21• 11 years ago

It depends on how far it will go. At the moment there is a plan how to do that, but not enough time to implement... You may use it for editor, but keep in mind that text itself would be sent to GPU every time it updates. Anyhow you may ping me in skype lp_funky if interested in doing this, would be glad to help.

perdunov8• 11 years ago

As far as I understand, this is roughly how iOS UI works (UIView - CALayer).

Obviously, this can be done at the browser engine level.

I have also tried to do this in my pet UI engine, but found out that it was tricky to figure out caching rules to avoid memory blowup and performance degradation for all possible cases.

PixelsCommander21• 11 years ago

It is done but there is no control over rasterization and no possibility to add filters as in the demo http://pixelscommander.com/polygon/htmlgl/demo/filters.html

perdunov8• 11 years ago

Kudos for the smart solution.

This approach has proven itself very well in iOS, in fact, it was the absolute best mobile animation technology for a long time. Actually, the older versions of iOS used software rasterization with hardware accelerated bitmap caching and it worked smoothly on the original iPhone.

cturhan3• 11 years ago

What if we merge it with React and use its virtual Dom management?

PixelsCommander21• 11 years ago

I think could be nice. There is an intention to create React-GL, but nothing serious yet. All you basically need is to call something like new GLElement(this.getDOMNode()). Or how is it in 0.13?...

cturhan3• 11 years ago

Actually there is an implementation called react-canvas does the same thing on canvas. Flipboard uses it on mobile website and it has a great performance even for mobile.

phn3• 11 years ago

Make it feature complete and you'll get the same speed the browser has.

If we're going to build a new rendering engine for the web, maybe we could try to create something nicer than HTML and CSS, whose only advantage at the present is the number of developers familiar with it. I know I wouldn't miss them.

emodendroket3• 11 years ago

What is there that's so much better?

aout7• 11 years ago

Because you could do something easier to understand in terms of 3D for a very simple use case and find a generic use for it.

moron4hire3• 11 years ago

There really isn't anything "better". HTML isn't great, but it's all we got right now.

le sigh.

kelvin0• 11 years ago

Postscript. Been there for ages and works like a charm.

i_s• 11 years ago

Almost any native UI framework (iOS, Android, WPF on windows, etc) has better layout options. Some of them like iOS have options that are not as beginner friendly (AutoLayout), but they are still better overall if you are trying to develop an application instead of formatting a document.

Many other UI frameworks have had Flexbox type layouts, a technology W3C has only just recently realized is a good idea for 8+ years (probably much longer).

icedchai• 11 years ago

Pretty much anything? HTML/CSS/JS was never meant to develop application UIs. It's for documents.

This all evolved over time into the clusterf-ck we have today, setting us back probably 20+ years in terms of developer productivity.

PixelsCommander21• 11 years ago

It is not a new rendering engine. It is just one custom tag which makes WebGL texture of your HTML content and allows to apply shaders. That is it...

phn3• 11 years ago

I am sorry if I sounded rude, I didn't mean to knock this particular project down.

I was commenting on a hypothetical future for something of its kind, and that I think HTML content is not really the way to go to make shiny WebGL things, IMO.

To be a little more on-topic, yes, as an "apply shader to this semi-static div" it is a pretty cool thing :)

_cpancake• 11 years ago

It uses html2canvas which is its own rendering engine to an extent.

shimonamit• 11 years ago

Not necessarily. Looks like it is an immediate mode, HTML-like syntax but without the DOM.

paraboul2• 11 years ago

FWIW, I'm working in this : http://www.nidium.com/ since a few years. (It's almost ready and about to be released)

Previous HN thread : https://news.ycombinator.com/item?id=6314961

video : https://www.youtube.com/watch?t=63&v=BSzEZXUTXlQ

voctor2• 11 years ago

It looks very interesting ! It will be open source ?

perdunov8• 11 years ago

Yep, the model of HTML-CSS-DOM-JS implies extreme overhead, and only heroic efforts and thousands of man-years of work and optimization have made it bearable.

I'm saying this as I have compared the performance of a simple local web page with a simple view with the same exact functionality rendered with my pet naive unoptimized UI engine and the latter was blazingly fast compared to HTML.

So I guess you are right, we definitely need something less cluttered and more efficient than HTML+CSS+JS.

BurningFrog• 11 years ago

Yeah, it feels like we're slowly approaching what we could do with native programs 20 years ago.

angersock3• 11 years ago

Compare how easy it is to make web pages with just HTML+CSS+JS, especially for a beginner, with any other framework, native code or no. The separation of structured markup, display theming, and custom logic is remarkably powerful, and everyone saying otherwise is just plain wrong if you look at history.

Once you start doing anything outside of your simple page with a simple view (which was what, incidentally?), your engine starts to accrue features that move it in "bloat" towards a modern web browser.

The link mentions "Hey, this is really useful on mobile"--and I would consider that a failing of the mobile vendors browser teams, not a problem with the web.

This looks really flashy for the sort of static content + basic animations + interactions the author is showing off, but once you start handling further interaction--like, you know, building a virtual dom to render to the webgl texture, all you're doing is reinventing the facilities the browser already gives you, but instead behind one more layer of abstraction.

tl,dr; web developers never learn.

macNchz• 11 years ago

HTML and CSS, whose only advantage at the present is the number of developers familiar with it

Given the sheer number of people familiar with it, I'd say this is a pretty huge advantage.

macspoofing2• 11 years ago

Make it feature complete

That's the opposite direction you want to go. At that point, you might as well just use HTML. I think there are benefits to providing a subset of HTML features that you can tailor for super fast performance.

PixelsCommander21• 11 years ago

The only bottleneck here is rasterization which we are trying to solve by making native Rasterization API live. You can read about here http://pixelscommander.com/en/javascript/state-of-html-conte...

moron4hire3• 11 years ago

This creates an interesting change in how applications are deployed to users. Instead of the user choosing the HTML rendering engine, the developer now gets to choose it. Instead of having to design sites with multiple rendering engines in mind, this makes it possible to manage only one. And it's not like the user cares that content is being rendered by Firefox or Chrome or whatever, the user just cares to see the page, leaving the selection of the actual rendering engine only of concern to the developer.

I think that's important enough to warrant going through the paces and implementing this. Don't listen to the naysayers. Reinvent wheels. The wheels we currently have largely suck.

perdunov8• 11 years ago

C++ with asm.js + WebGL effectively bring back desktop programming to the web.

moron4hire3• 11 years ago

That is so not even close to a good way to build applications. I mean, I really can't think of anything more painful right now. Other than, maybe being asked to make a modern web application on classic ASP.

pierrec• 11 years ago

Very cool. This could be useful for some applications, but I don't see it becoming a thing for general use cases unless it's implemented and exposed directly by the browsers.

Even then, the main problem is that the internet is still mostly text, and if you can't implement or access font hinting, your text will be blurry and ugly on any display that isn't ultra-high DPI. You can't really render text to a texture and then manipulate that texture with sub-pixel precision. It must be done the other way around: text rendering must be done last, and with the help of a hinting library so it will look sharp and readable.

PixelsCommander21• 11 years ago

There is an intent to make rasterization native http://pixelscommander.com/en/javascript/state-of-html-conte...

SoapSeller• 11 years ago

This project(at least for me) is a little bit misleading. Because it isn't rendering HTML using WebGL.

It using html2canvas[0] to rasterize the browser rendering into a canvas, upload it to WebGL, apply effects and patch-back events(i.e. mouse clicks) back to the hidden browser dom elements. (On the other side, font hinting and such are not really a problem)

Implementing full UI framework over WebGL has it merits, and hopefully someone will make a useful(and not html+css based) one some day.

[0] https://github.com/niklasvh/html2canvas

Polarity• 11 years ago

u know http://famo.us ?

kowdermeister• 11 years ago

Famous is CSS based, not WebGL

j1e• 11 years ago

^ really cool - thanks!

fapjacks3• 11 years ago

Yet another instance of html2canvas being used to do something interesting. It's the library I have been surprised by more than any other, for any language.

macspoofing2• 11 years ago

Implementing full UI framework over WebGL has it merits, and hopefully someone will make a useful(and not html+css based) one some day

I'm surprised it hasn't been done already (in similar spirit to the Feathers framework for Flash's Stage3D). Now that WebGL is supported in Mobile WebView, maybe you'll see a really good component set released? That would give devs a really slick option for building smooth phone-gap based apps.

josephmx• 11 years ago

Very minor bug on the demo: The alert on click reads "clicked X" (eg, "Clicked Avatar") on all of the titles except for Quantum of Solace

curiously• 11 years ago

this idea is so crazy it just might work but need to see some more examples.

one might be a crazy heavy SPA or javascript webpage that lags and show it through HTMLGL.

this really could go somewhere, the idea is solid but it might take some work to actually deliver, a constant fps web.

PixelsCommander21• 11 years ago

Yes, the only bottleneck here is rasterization which we are trying to solve by making native Rasterization API live. You can read about here http://pixelscommander.com/en/javascript/state-of-html-conte...

slaction• 11 years ago

"I really wish I could use bump mapping on this div" - No one ever

mandeepj• 11 years ago

Then all the hard work that flipboard did to achieve 60 fps rendering speed was not required?

Reference - https://news.ycombinator.com/item?id=9029159

Render HTML via WebGL

Related Stories