A Bayesian Approach for Selective Image-Based Rendering using Superpixels

Image-Based Rendering (IBR) algorithms generate high quality photo-realistic imagery without the burden of de-tailed modeling and expensive realistic rendering. Recent methods have different strengths and weaknesses, depending on 3D reconstruction quality and scene content. Each algorithm operates with a set of hypotheses about the scene and the novel views, resulting in different quality/speed trade-offs in different image regions. We present a principled approach to select the algorithm with the best quality/speed trade-off in each region. To do this, we propose a Bayesian approach, modeling the rendering quality, the rendering process and the validity of the assumptions of each algorithm. We then choose the algorithm to use withMaximum a Posteriori estimation. We demonstrate the utility of our approach on recent IBR algorithms which useover segmentation and are based on planar reprojection and shape-preserving warps respectively. Our algorithm selects the best rendering algorithm for each superpixel in a preprocessing step; at runtime our selective IBR uses this choice to achieve significant speed up at equivalent or better quality compared to previous algorithms.

@InProceedings{ODD15,
  author       = "Ortiz-Cayon, Rodrigo and Djelouah, Abdelaziz and Drettakis, George",
  title        = "A Bayesian Approach for Selective Image-Based Rendering using Superpixels",
  booktitle    = "International Conference on 3D Vision (3DV)",
  year         = "2015",
  publisher    = "IEEE",
  key          = "Image-based rendering, image warp, superpixels, variational warp, wide baseline, multi-view stereo",
  url          = "http://www-sop.inria.fr/reves/Basilic/2015/ODD15"
}