ADOP: Approximate Differentiable One-Pixel Point Rendering

https://arxiv.org/abs/2110.06635

by rohithkp • 5 years ago

45 6 5 years ago

arxiv.org

Luc• 5 years ago

Video presentation: https://www.youtube.com/watch?v=WJRyu1JUtVw

MauranKilom2• 5 years ago

Thank you, this gave me a very good introduction to the work!

MauranKilom2• 5 years ago

So, if I understand correctly, they have created an algorithm that takes several colored point clouds (or photos + depth maps) of a scene and can generate renderings of the scene from novel viewpoints (or with different camera parameters, e.g. exposure or white balance). The results look truly impressive!

The whole rendering pipeline is differentiable, so they can backpropagate the ground truth error to all the unknowns in the pipeline. Those include values like camera pose parameters, point cloud texture and point positions, weights of a neural network that turns (potentially sparse) point cloud rasterizations into full HDR (high dynamic range) images, and tone mapping parameters like exposure.

(If I understand correctly, they start from something like a SLAM-based point cloud, but I haven't found that explained in glancing over the paper.)

Nitpick for the paper: A brief note on what HDR and LDR mean would be useful. The acronyms are not even expanded anywhere.

me_again• 5 years ago

High and Low Dynamic Range

dougabug• 5 years ago

Structure from Motion (SfM) generated point clouds. Typical SLAM point clouds are much sparser so they can be run in real-time.

mekkkkkk• 5 years ago

As I'm interested in the subject, I immediately think about the possibility of point cloud tech in a game development context. As far as I know, the big hurdle has always been dynamism like animation/deformation. Does this approach have any influence on that aspect?

ADOP: Approximate Differentiable One-Pixel Point Rendering

Related Stories