Filament: Physically-based rendering engine

https://google.github.io/filament/Filament.md.html

by corysama • 8 years ago

599 125 8 years ago

google.github.io

corysama4• 8 years ago

The actual code: https://github.com/google/filament

Additional docs on their material system:

https://google.github.io/filament//Materials.md.html

https://google.github.io/filament/Material%20Properties.pdf

Also, I'll plug http://casual-effects.com/markdeep/ which was apparently used to format the docs.

yters• 8 years ago

Really beautiful. Funny how high tech 3D engines get better at making things that look low tech and run down.

taneq• 8 years ago

It's kind of like Moravec's paradox but for graphics. :)

buchanae4• 8 years ago

Wow. It's so hard to find expert-level documentation like this all in one page these days. I'm usually stuck piecing together information from a couple dozen (often poorly presented) websites. The topic is fantastic, but the presentation deserves some applause too!

RomainGuy9• 8 years ago

Glad you like it. We found it hard to piece together all the information we needed and after reading hundreds of papers and presentations we figured that writing such a document would be helpful to beginners like us (in this field that is).

buchanae4• 8 years ago

I think my render in Chrome is pretty broken. Here's a screenshot of the top of the page: https://drive.google.com/open?id=1YVtcV-J0JrDyW2pJ6vwgEDsgjO...

valine• 8 years ago

The PBR tutorial series on LearnOpenGL.com covers a lot of the concepts implemented here like physically based BRDFs and HDR lighting, if anyone is looking for more resources like this.

https://learnopengl.com/PBR/Theory

corysama4• 8 years ago

If like me you actually enjoy tech talk videos, "SIGGRAPH University - Introduction to Physically Based Shading in Theory and Practice" is great. PDF version is linked at the bottom of your learnopengl page. https://www.youtube.com/watch?v=j-A0mwsJRmk

gerdesj3• 8 years ago

Ahhhhhhhhhhh - I will be DVd to death for this but:

I have a screen (it happens to be about 18" wide in this case). Why on earth is the text in a stupidly thin column in a small font? It looks like TeX's daemonic alter thingie got into the render process somewhere or perhaps someone forgot the other two columns or wanted to torture a webby reader with multiple columns (mmmm ArXiv scrolly uppy n downy pdf) and lost interest.

Anyway ... it looks crap in my browser unless I hit CTRL(num)+ a few times - 200% works.

RomainGuy9• 8 years ago

Would you mind sharing a screenshot somewhere? I'm just using Markdeep's default CSS and the font size seems reasonable on all my screen. I'd be happy to try and tweak it though.

gerdesj3• 8 years ago

I'm not sure what has gone wrong (if anything) - I'm a sysadmin not devops/webby minded.

This is what I see:

https://nextcloud.roseandjon.gerdes.co.uk/nextcloud/index.ph... - that should yield two images.

Fonts/typefaces - lovely. Layout - a bit limited.

mcbits• 8 years ago

The text seems perfectly sized for reading on my screen. At 200% it looks like one of those clownish Bootstrap homepages and I'd have to scroll every 5 seconds to read it.

leetbulb• 8 years ago

Agreed. Looks fine here at 27" 2160p (no scaling) and on my 13" 900p macbook air

anonytrary3• 8 years ago

That is done on purpose. Ideally, one's eye's horizontal angle span will be less than 30 degrees when one is reading something. WPM drops if the eyes or neck have to do too back-and-forth work.

The brain doesn't require the eyes to hit every single word to extract the full text thanks to peripheral vision, but this optimization is not as effective when the width of the text-line is too large.

I am surprised you didn't mention the FOUC, though. For a while you see raw, minified, un-styled text on this page.

rangibaby• 8 years ago

It looks good and I had no trouble reading it on my phone. It seems to follow the rule I learned of 7-10 words per line.

ArtWomb• 8 years ago

Kudos to Romain Guy on the Android Graphics team. Stunning WebGL demo here:

http://www.curious-creature.com/2017/08/14/physically-based-...

Good example of visual state-of-the-art on Android is probably the ARK dinosaur game. The Vulkan API is now included on latest Android. And with the compact gltf 2.0 format we can expect great experiences on mobile ;)

Vulkan glTF 2.0

https://www.youtube.com/watch?v=sl7iN-vQCOs&list=PLy80eMh1-z...

RomainGuy9• 8 years ago

The WebGL demo was the initial prototype of Filament :)

kkimdev• 8 years ago

Quick question: Is WebGL support dropped in the final release?

naikrovek5• 8 years ago

glTF is pretty great, to be honest. At work I am writing a tool to convert Creo View models to glTF 2.0 models and I am experiencing actual joy while doing it.

I looked at the COLLADA and glTF formats (I wanted something open) and very quickly chose glTF 2.

I don't have any 3D format implementation experience, and I implemented glTF 2 in Golang in a day, or less.

Anyway, it's a great format (AFAIK) and I wish more 3D modeling tools supported it. I imagine more will come on board as time passes.

timdorr• 8 years ago

Any thoughts on these criticisms from this Wikipedia article? https://en.wikipedia.org/wiki/GlTF#Criticism

(I'm not agreeing or disagreeing with them, I'm genuinely curious to hear from someone working with the format at the implementation level)

corysama4• 8 years ago

It's important to separate the idea of editor formats vs. delivery formats. COLLADA/FBX vs glTF is very analogous to Photoshop PSD vs. PNG. Even if you could, you wouldn't fill a website with PSD-format images, right? Right? But, there have been misguided WebGL apps that tried delivering COLLADA/OBJ/STL files to web pages.

It's understandable. Before glTF, there really wasn't a good non-secret/proprietary format with which to deliver 3D content. But, now there is! Use it!

sydd• 8 years ago

any mirror for the WebGL demo? its down

dwaite• 8 years ago

AFAIK Vulkan is still optional in latest Android.

pjmlp• 8 years ago

Vulkan might be included on Android, but good luck getting a version that actually works across all devices.

https://vulkan.gpuinfo.org/vulkansupport.php#android_devices

ChuckMcM2• 8 years ago

Ok this is way fun and drags me back to the days I was diligently trying to build a 3D rendering engine from first principles (sort of, I had Glide to put stuff on the screen).

The renderer is perceptually better than the one that is included in my CAD package (TurboCAD) for pretty much all materials. So I'm guessing they will snarf it and dump the proprietary renderer and replace it with this stuff if they can.

But the really interesting idea that popped into my head was this; could Google offer 'rendering as a service'?

Specifically they have a zillion machines, many of which are doing nothing important, and they have this rendering package, and they have a scheduler that can put things on any machine. Imagine a service where you sent them a suitably detailed model description, and a set of lights, could they send you back a rendered image? Could you parameterize changes to the model description over time so that they could send you images in time based on your models? Could they do say 480 renders 'free' per month and then maybe $0.19/render over 480 in a single month?

Could you create a studio of modellers who would design models, and animators that would animate those models over time, and a director who would compose those animations into scenes? This is basically Pixar without the expensive renderfarm. Does that enable new studios to bring their own vision to life? Does it offer a cost effective service to places like Pixar which allows Google to make money on otherwise idle resources? Curious minds want to know :-)

steren• 8 years ago

Google Cloud Platform offers https://www.zyncrender.com/

ChuckMcM2• 8 years ago

Okay that is pretty close, and acquired by Google in 2014. So presumably it is possible to be a small CGI shop and use this as your back end. Now I'm wondering if the economics pencil out. Clearly there was something that motivated Google to buy them.

Gravityloss• 8 years ago

Of course. With Fusion 360 you can have your stress calculations done in the cloud.

eat_veggies• 8 years ago

Lagoa [0] used to do cloud based rendering as a service. But Autodesk bought them out and shut them down.

[0] https://web.archive.org/web/20171017063005/http://home.lagoa...

luma• 8 years ago

OTOY has been doing this for years, and now working on a blockchain approach to utilize spare GPU cycles from third parties who have cycles to spare.

https://home.otoy.com/render/orc/

socratesque• 8 years ago

I don't know how useful it really is at this point, but Golem* currently offer rendering as a service.

* https://golem.network/

cloud_man• 8 years ago

I've used Golem and it works well and is cheap. Only issue is buying GNT which is a pain.

mrguyorama• 8 years ago

I don't think a large rendering farm is as expensive as a huge amount of animators and artists who are talented enough to saturate the render farm.

However I have no knowledge in the industry, just a gut feeling

fallingfrog• 8 years ago

I'm still waiting for someone to do a completely physically based renderer, and simulate the whole EM spectrum, not just the 3 colors we can see. Then you can model chromatic aberration, and the difference between fluorescent and incandescent lighting, and prisms.. you could call it a really really physically based rendering engine.

modeless• 8 years ago

Here you go: https://www.indigorenderer.com/unbiased-rendering

tlarkworthy• 8 years ago

Wow those diamonds and sky. I think I might be fooled by the diamonds

RomainGuy9• 8 years ago

There are several spectral renderers out there, such as Weta Digital's Manuka. I don't know if they bother with parts of the EM spectrum that are outside of the visible range though. I imagine UVs can be important to model in some situations.

mattkrause2• 8 years ago

I think Mitsuba can be compiled to use (binned) spectra instead of RGB.

berkut2• 8 years ago

To model fluorescence accurately, it's necessary to handle the non-visible spectrum in some way.

arketyp3• 8 years ago

I've long wondered how the world would appear if we had a different set of cone cell photoreceptors and if a full spectrum engine could be used to simulate that by projecting a different set of spectrum ranges to our eyes. It would still be "seeing in RGB" of course, but entirely different aspects of the world would become visible. I can already do this when converting an image to grayscale, giving different luminance weight to the different channels. It would be really interesting to arrange weights on the entire EM spectrum.

(I'm reminded of some company that developed a pair of glasses for color-blind people which enabled them to see "color". I think it was a similar princple.)

ttoinou5• 8 years ago

Multiple easy-to-understand things will happen :

- colors that were previously the same could be now perceived different (a red car, a red flower, red light from LCD screen etc...)

- White / grey would still exist but be less common, as a lot of white / grey light would now be close to a new receptor

- Probably more distinguishable shades in the rainbow

alkonaut• 8 years ago

Spectral renderers are pretty common. A naive implementation just assigns a random wavelength to the ray when tracing, which results in surprisingly little extra code to achieve all spectral effects such as abberation.

The downside is that this is slow. An RGB-ray traces the whole spectrum for one ray (approximated by an RGB triplet). The single frequency ray does not. If you try to send a binned spectrum in the ray, what will happen is that as soon as you hit something refractive, they will go in different directions, so you no longer have a full spectrum. All the complexity in a spectral renderer over an RGB one lies in handling performance: How to avoid the performance problems when you lose coherence. Apart from that it’s actually often simpler than an RGB renderer despite being able handle all those fancy spectral effects.

For a nice “reference” or “tutorial” one that isn’t plagued by the performance hacks needed to make it viable for large scenes, look at this one: https://github.com/TomCrypto/Lambda

Solar192• 8 years ago

This looks very good.

Ironic aside: Notice the "Processing math" message on the lower left when you first open the page? That's MathJax, a huge JS library, chugging away because Chrome doesn't support MathML. Try going to the page in Firefox and you'll see that it processes "the math" a hell of a lot faster than Chrome. Firefox supports MathML, and MathJax probably generates MathML for it.

buchanae4• 8 years ago

Oh, I actually didn't realize how much of the page I was missing until I opened this in Firefox. My page is very broken in Chrome 67.0

stepik777• 8 years ago

No it doesn't, it renders it as a bunch of spans. MathML is basically a dead standard for the web - browsers either don't support it or support is incomplete and with bugs which makes it useless.

https://www.peterkrautzberger.org/0186/

shawn9• 8 years ago

There is no such thing as physically based rendering, and the sooner the world learns this the better. It’s a marketing term, nothing more.

Here’s food for thought: what does it mean to multiply two colors? Nothing. It’s meaningless. It’s an approximation that happens to look good. But if you’re going to claim your engine is physically based, you can’t use it, because multiplying two colors is not based in physics.

pandaman6• 8 years ago

"Physically based" in context of rendering means just that the light calculations preserve energy of light. So there is such thing actually.

shawn9• 8 years ago

There is no clear definition, further adding to the confusion.

https://en.wikipedia.org/wiki/Physically_based_rendering

https://marmoset.co/posts/basic-theory-of-physically-based-r...

sometimesijust• 8 years ago

That is a description of unbiased rendering.

taeric2• 8 years ago

This doesn't seem to be a substantive criticism. Abstractions, even those that are for physics, are done that way as far as it is convenient. Those by physics are the same.

Similarly, what does it mean to add colors? Well, kind of non sequitur. Depends entirely on what abstractions you are working on.

shawn9• 8 years ago

If it isn't a substantive criticism, then it should be easy to address.

The claim is precise: It's meaningless to multiply two colors, yet this is the core abstraction that we use in our rendering engines.

New ideas are needed to advance the state of the art.

phiresky2• 8 years ago

You should look more into how physically based rendering works.

You do actually approximate the energy that reaches the different surfaces (in Watt / (m² sr)) for all wavelengths of light using Monte-Carlo integration. You can send probes for different wavelengths separately if you care about the differences in the refraction index of your material by wavelength.

Then you integrate the results by folding it with the corresponding color response of the primaries of sRGB or whatever color space you want to use. The color representation can be accurate, though screens etc are obviously limited.

shawn9• 8 years ago

This is unrelated to the claim that multiplying two colors is meaningless.

ttoinou5• 8 years ago

I agree that we should study more in-depth the "physic" adjective in the name of the rendering, but what's the argument exactly against multiplying colors ? Isn't it an equivalent of a light absorption ?

Solar192• 8 years ago

Also, there should be a warning in case you're running tab heavy on a lower end machine, or just out of courtesy. The web page is 37 MB (saved as an MHTML in Chrome -- see your About:Chrome flags). It's yuge. MathJax alone is usually >1 MB of JavaScript.

malkia• 8 years ago

Had to do this:

CC=clang CXX=clang++ ./build.sh -j release

to get it started to compile (as it seems it had cc pointing to gcc and it did not understand some, apparently clang-specific flags).

andybak5• 8 years ago

1. What's the likelihood of seeing a Unity wrapper for this in my lifetime? I imagine the takeup of it if it plugged into Unity or Unreal would be massively increased.

2. How does it compare with what OTOY are working on for realtime? I imagine they are focused on quality over performance and aren't even considering mobile.

kowdermeister3• 8 years ago

PBR is already enabled in most game engines, it's even supported out of the box by Three.js

andybak5• 8 years ago

I'm asking about this specific engine - not PBR in general.

lebek• 8 years ago

Unity and Unreal both support Android and PBR so I don't see an obvious benefit to being able to use this alternative renderer. Because of the license and the fact that it's a newish codebase written in simple C++ I can see it being popular as a starting point for in-house engines.

clankfan• 8 years ago

Does anyone know how this document relates the the rendering done in the 2016 ratchet and clank game? That game has always astounded me and 8ve never been able to find a deep and comprehensive relation about how the rendering works.

RomainGuy9• 8 years ago

It's similar techniques, just better because they can afford more effects on a PS4 :)

forgotpwd16• 8 years ago

It should provide adequate information as R&C 2016 is using cartoonish stylized pbr and many concepts/methods are common among different engines. Though most pbr engines seem to target photorealistic needs, physically rendered materials and lighting make even non-realistic models and environments look better.

corysama4• 8 years ago

They certainly use some of these techniques.

Here’s a bit of convo about the code and art of R&C

https://youtube.com/watch?v=Y65h1aO-xps

https://youtube.com/watch?v=zpvXB4yWvyA

http://advances.realtimerendering.com/s2014/index.html

hi41• 8 years ago

Such beautiful documentation!

skavi• 8 years ago

Do will this replace Escher? Or do they serve different purposes? (UI vs 3D)

Filament: Physically-based rendering engine

Related Stories