GraphDeco

Multi-View Inpainting for Image-Based Scene Editing and Rendering
Presentation | Team members | Collaborations | Publications | Job offers | Contact

 

Multi-View Inpainting for Image-Based Scene Editing and Rendering

Theo Thonat, Eli Shechtman, Sylvain Paris, George Drettakis
International Conference on 3D Vision (3DV) - 2016
Download the publication : final_MVInpainting.pdf [1.8Mo]  
We propose a method to remove objects such as people and cars from multi-view urban image datasets, enabling free-viewpoint Image-Based Rendering (IBR) in the edited scenes. Our method combines information from multi-view 3D reconstruction with image inpainting techniques, by formulating the problem as an optimization of a global patch-based objective function. We use IBR techniques to reproject information from neighboring views, and 3D multi-view stereo reconstruction to perform multi-view coherent initialization for inpainting of pixels not filled by reprojection. Our algorithm performs multi-view consistent inpainting for color and 3D by blending reprojections with patch-based image inpainting. We run our algorithm on casually captured datasets, and Google Street View data, removing objects such as cars, people and pillars, showing that our approach produces results of sufficient quality for free-viewpoint IBR on ``cleaned up'' scenes, as well as IBR scene editing, such as limited displacement of real objects.

Images and movies

 

See also

Acknowledgements and Funding

The authors are grateful for the support of EU project CR-PLAY (no 611089) www.cr-play.eu , donations from Adobe, and project SEMAPOLIS (ANR-13-CORD-0003) and the doctoral fellowship of the Region Provence Alpes Cote d'Azur.
We also thank Sebastien Bonopera for implementing the multi-view segmentation and IBR editing application.

Video

BibTex references

@InProceedings{TSPD16,
  author       = "Thonat, Theo and Shechtman, Eli and Paris, Sylvain and Drettakis, George",
  title        = "Multi-View Inpainting for Image-Based Scene Editing and Rendering",
  booktitle    = "International Conference on 3D Vision (3DV)",
  year         = "2016",
  url          = "http://www-sop.inria.fr/reves/Basilic/2016/TSPD16"
}

Other publications in the database

» Theo Thonat
» Sylvain Paris
» George Drettakis