Interactive audio/video
rendering pipelines
Part of my PhD was dedicated at creating
an integrated audio and video rendering system that allows for interactive
production of synthetic animations with synchronized sound.
Based on an image-sources model,
the developed sound rendering pipeline can simulate perfectly synchronized
sound together with a 3D visual animation.
It takes into account :
-
source, microphones, materials directional
filtering
-
Doppler effect,
-
atmospheric scattering
-
transmission/diffraction
The system simulates the propagation paths
from an arbitrary number of point sources (including the image-sources)
to an arbitrary number of point receivers. Each path is processed using
resampling of the source signal and short FIR convolution
with the path impulse reponse. At the end of the process, contributions
of each receiver can be mapped to the selected restitution system
(mono, stereo, stereo binaural, stereo transaural, matriced, multi-channel)
using a corresponding mixing engine.
Related
publications