GraphDeco

Point-Based Neural Rendering with Per-View Optimization
Presentation | Team members | Collaborations | Publications | Job offers | Contact

 

Point-Based Neural Rendering with Per-View Optimization

Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering), Volume 40, Number 4 - June 2021
Download the publication : KopanasPointBasedNeuralRenderingPerViewOptimization.pdf [12.5Mo]   SupplementalKopanasPointBasedNeurealRendering.pdf [2.5Mo]  
There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel view synthesis. A key element of our approach is a differentiable point-based splatting pipeline, based on our bi-directional Elliptical Weighted Average solution. To further improve quality and efficiency of our point-based method, we introduce a probabilistic depth test and efficient camera selection. We use these elements together in our neural renderer, allowing us to achieve a good compromise between quality and speed. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel view synthesis.

Images and movies

 

See also




See also the project webpage. Full source code and datasets will be provided soon.

Acknowledgements and Funding

This research was funded in part by the ERC Advanced grant FUNGRAPH No 788065 (http://fungraph.inria.fr). The authors are grateful to the OPAL infrastructure from Université Côte d'Azur for providing resources and support. Finally, the authors thank the anonymous reviewers for their valuable feedback.

BibTex references

@Article{KPLD21,
  author       = "Kopanas, Georgios and Philip, Julien and Leimk{\"u}hler, Thomas and Drettakis, George",
  title        = "Point-Based Neural Rendering with Per-View Optimization",
  journal      = "Computer Graphics Forum (Proceedings of the Eurographics Symposium on Rendering)",
  number       = "4",
  volume       = "40",
  month        = "June",
  year         = "2021",
  url          = "http://www-sop.inria.fr/reves/Basilic/2021/KPLD21"
}

Other publications in the database

» Julien Philip
» Thomas Leimkühler
» George Drettakis