GraphDeco

Image-Based Rendering of Cars using Semantic Labels and Approximate Reflection Flow
Presentation | Team members | Collaborations | Publications | Job offers | Contact

 

Image-Based Rendering of Cars using Semantic Labels and Approximate Reflection Flow

Proceedings of the ACM on Computer Graphics and Interactive Techniques, Volume 3, Number 1 - may 2020
Download the publication : rodriguez2020-ibr-cars-semantic-and-reflection.pdf [16.5Mo]   rodriguez2020-supplemental.pdf [4Mo]  
Image-Based Rendering (IBR) has made impressive progress towards highly realistic, interactive 3D navigation for many scenes, including cityscapes. However, cars are ubiquitous in such scenes; multi-view stereo reconstruction provides proxy geometry for IBR, but has difficulty with shiny car bodies, and leaves holes in place of reflective, semi-transparent windows on cars. We present a new approach allowing free-viewpoint IBR of cars based on an approximate analytic reflection flow computation on curved windows. Our method has three components: a refinement step of reconstructed car geometry guided by semantic labels, that provides an initial approximation for missing window surfaces and a smooth completed car hull; an efficient reflection flow computation using an ellipsoid approximation of the curved car windows that runs in real-time in a shader and a reflection/background layer synthesis solution. These components allow plausible rendering of reflective, semi-transparent windows in free viewpoint navigation. We show results on several scenes casually captured with a single consumer-level camera, demonstrating plausible car renderings with significant improvement in visual quality over previous methods.

Images and movies

 

See also




See also the project webpage.

NEW!!!! Source Code and Datasets

Full source code and datasets are available at https://gitlab.inria.fr/sibr/projects/semantic-reflections/semantic_reflections as part of the SIBR system. For full documentation see https://sibr.gitlabpages.inria.fr/ .

Acknowledgements and Funding

The authors would like to thank S. P. Bangaru for initial tests, G. Brostow for fruitful discussions, J. Philip and S. Morgenthaler for helping with the supplemental material. This research was funded by the ERC Advanced Grant FUNGRAPH No 788065 (http://project.inria.fr/fungraph) and by the Rabin Ezra scholarship fund. The authors are grateful to Adobe for software donations and to Inria Sophia Antipolis - Mediterranée "Nef" computation cluster for providing resources and support.

BibTex references

@Article{RPHD20,
  author       = "Rodriguez, Simon and Prakash, Siddhant and Hedman, Peter and Drettakis, George",
  title        = "Image-Based Rendering of Cars using Semantic Labels and Approximate Reflection Flow",
  journal      = "Proceedings of the ACM on Computer Graphics and Interactive Techniques",
  number       = "1",
  volume       = "3",
  month        = "may",
  year         = "2020",
  keywords     = "image-based rendering, semantic labeling, reflection rendering",
  url          = "http://www-sop.inria.fr/reves/Basilic/2020/RPHD20"
}

Other publications in the database

» Simon Rodriguez
» Siddhant Prakash
» Peter Hedman
» George Drettakis