ADDITIONAL Material
Retargetting Example Sounds
to Interactive Physics-Driven Animations
Overview
We present the results of our method to generate audio in the context of interactive animations driven by a
physics engine, such as those used extensively in games. Our approach consists in retargetting
audio grains extracted from pre-recorded contact sounds to interactive physics-driven
animations.
We first present results of our approach for resynthesis. In this case, the purpose is simply to reduce the size
of the recording database manipulated during real-time interaction in
video games. The method is applied on recorded
contact events involving different material types (e.g. stone glass,
metal, wood, etc..) and interaction types (e.g. break, hit, roll,
slide). We also demonstrate that our technique is well adapted for the specific case of
continuous-like events such as rolling or sliding. A specific operation is performed
on continuous-like recordings that separates the sinusoidal and the transient parts to allow coding them
separately.
Using this approach, the audio grains can be efficiently concatenated using the coding structure of user-defined, shader-like
procedures. Similar audio events can be built using different approaches.
Time-scale modification of the original recordings is easily implemented acting on the Time Modification factor (TM); TM<1 induces shrinking whereas TM>1 induces stretching. Different examples are given (1).
Besides time scaling, continuous-like recordings are varied by enhancing the sinusoidal part or the transient part extracted in a preprocess analysis step, acting on the granularity sensation of the contact interaction (2).
Finally, variable audio content is appropriately matched to a caracteristic rhythmic
pattern of an available audio recording using cross-correlations calculations(3).
Finally, we present our results for an interactive video sequence, similar to a game setting. Here atom
distribution is tailored by parameters from game data. Sounding material is retargeted to simulated sounding events through the
amplitude of the normal force and relative velocity parameters.
Each sound clip contains the original exemplar followed by the synthesis result.
Resynthesis Results
STONE Interactions
Total Resynthesis Encoding Size (vs original) = 80%
IMPACT-like contacts
. Stone/Break (30 records): 1034 impulsive atoms
Stone/Break events 1
2
3
. Stone/Hit (44 records): 1086 impulsive atoms
Stone/Hit events 1
2
3
CONTINUOUS-like contacts
. Stone/Roll (3 records): 276 impulsive atoms
- 313 continuous atoms
Stone/Roll events 1
2
. Stone/Slide (17 records): 1250 impulsive atoms
- 1212 continuous atoms
Stone/Slide events 1
2
Stone/Gun events 1
2
WOOD Interactions
IMPACT-like contacts. Wood/Break events 1 2 . Wood/Hit events 1 2
CONTINUOUS-like contacts
. Wood/Roll events 1 2 3 4
GLASS Interactions
IMPACT-like contacts. Glass/Break events 1 2 . Glass/Hit 1 2
CONTINUOUS-like contacts
. Glass/Roll 1 2 3 4
Results of Similar Patterns Synthesis
TIME-SCALE Modifications
. TM=0.3 (shrinking): 1 2 3 . TM=1.3 (stretching): 1 2 3
CONTINUOUS-like contacts variations
independent time-scale (TM) and granularity (Gran) modifications1 : TM=1 - Gran=0.45 2 : TM=1 - Gran=4 3 : TM=0.3 - Gran=1
1 : TM=0.4 - Gran=2 2 : TM=1.2 - Gran=0.08 3 : TM=1.4 - Gran=0.4
Audio Content Variations
only the rhythmic pattern of the original recording is keptBreaking Glass 1 Breaking Glass 2 Breaking Stone/Ceramic 1
Breaking Stone/Ceramic 2 Breaking Stone Sliding Wood
