A Bio-inspired Synergistic Virtual Retina Model for Tone Mapping

Marco Benzi, Maria-Jose Escobar, Adrien Bousseau, Pierre Kornprobst

CVIU 2017 : Special Issue on Vision and Computational Photography and Graphics

Real-world radiance values span several orders of magnitudes which have to be processed by artificial systems in order to capture visual scenes with a high visual sensitivity. Interestingly, it has been found that similar processing happen in biological systems, starting at the retina level. So our motivation in this paper is to develop a new video TMO based on a synergistic model of the retina. We start from the so-called Virtual Retina model, which has been developed in computational neuroscience. We show how to enrich this model with new features to use it as a TMO, such as color management, luminance adaptation at photoreceptor level and readout from a heterogeneous population activity. Our method works for video but can be applied to static images seen as a video of a static frame. It has been carefully evaluated on standard benchmarks in the static case, giving comparable results to the state-of-the-art using default parameters, while offering user control for finer tuning. Result on HDR video are also promising, specifically w.r.t. temporal luminance coherency. Code is available as a Python notebook and a C++ implementation through GitHub so that reader could test and experiment the approach step-by-step. As a whole, this paper shows a promising way to address computational photography challenges by exploiting the current research in neuroscience about retina processing.

Images

MemorialBristol BridgeWaffle HouseClock BuildingLuxo Double CheckerExploratoriumGolden GateLabBoothWall DrugCathedral MaskLampicka MaskTree Mask

Videos

Parameter sensibility of i1/2

Here there are some comparisons over the values of i1/2.

i1/2 = 0.01 i1/2 = 1 i1/2 = 100 i1/2 = 10000

Parameter sensibility of temporal constant tauA

No temporal filtering tauA = 0.0005 tauA = 0.005 tauA = 0.05

Code

Code for the latest version of our implementations can be found in our GitHub repository.

We provide two versions of the implementation: A python notebook tailored for HDR image processing, allowing fast manipulation of parameters and a step-by-step approach of our model, and a C++ application for both images and videos, which is faster and has the temporal filtering enabled.


Memorial image is copyright of Paul Debevec; Bristol Bridge and Clock Building are copyright of Greg Ward; Waffle House, Luxo Double Checker, Exploratorium, Golden Gate, Lab Booth and Wall Drug are copyright of Mark D. Fairchild; and Cathedral Mask, Lampicka Mask and Tree Mask are copyright of Industrial Light and Magic.