Plane-Based Multi-View Inpainting for Image-Based Rendering in Large Scenes

Image-Based Rendering (IBR) allows high-fidelity free-viewpoint navigation using only a set of photographs and 3D reconstruction as input. It is often necessary or convenient to remove objects from the captured scenes, allowing a form of scene editing for IBR. This requires multi-view inpainting of the input images. Previous methods suffer from several major limitations: they lack true multi-view coherence, resulting in artifacts such as blur, they do not preserve perspective during inpainting, provide inaccurate depth completion and can only handle scenes with a few tens of images. Our approach addresses these limitations by introducing a newmulti-viewmethod that performs inpainting in intermediate, locally common planes. Use of these planes results in correct perspective and multi-view coherence of inpainting results. For efficient treatment of large scenes, we present a fast planar region extraction method operating on small image clusters. We adapt the resolution of inpainting to that required in each input image of the multi-view dataset, and carefully handle image resampling between the input images and rectified planes. We show results on large indoors and outdoors environments.

@InProceedings{PD18,
  author       = "Philip, Julien and Drettakis, George",
  title        = "Plane-Based Multi-View Inpainting for Image-Based Rendering in Large Scenes",
  booktitle    = "Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games",
  month        = "may",
  year         = "2018",
  organization = "ACM",
  keywords     = "Inpainting, Multi-View, Image-Based Rendering",
  url          = "http://www-sop.inria.fr/reves/Basilic/2018/PD18"
}