Contextual Information

Contextual information or context is an a priori information which contains a description of the static environment observed by a camera. We propose a common formalism to describe a context. For each particular camera looking at a particular environment, a human operator must in a configuration phase provide the pertinent information according to the formalism we propose. For more details on the role of context in video understanding see [5].

The context contains in addiction to of geometric information some semantic information. Its structure is made of a set of frames: one frame per physical object, one frame per interesting area and a calibration matrix which enables the passage from the 2D image plane to 3D coordinates.

The geometry of the interesting areas are described by a list of polygons defined in planes which may have any orientation. The geometry of each physical object, or piece of equipment, is a generalized cylinder defined by its height and its polygonal basis.

The semantic information of each piece of equipment and of each interesting area is made of six attributes : four with symbolic values and two with numerical values.

The four attributes with symbolic values are : the type of the object (equipment or area), the function of the object (i.e. table, seat, corridor, etc...) the name of the object (i.e. seat3, corridor2, etc...) and for some objects their characteristics (i.e. yellow, fragile, etc...) which can be used in the scenario recognition mechanism. The two attributes with numerical values which are very useful for scenario recognition are: the normal distance and the normal time of usage of a equipment.

Figures 2 and 3 show two examples of the environments we have modeled; for each example the left image shows the view observed from the camera and the right image shows the 3D description of the same environment based on the geometrical information contained in the context.

  

Figure 2: Coffee room context Left: images of empty scenes, Right: 3D scene models.

  

Figure 3: Nuremberg Metro Station Left: images of empty scenes, Right: 3D scene models.

For instance for the first example of a coffee room the context contains:

• the calibration matrix
• the description of nine pieces of equipment: three seats, one table, one coffee machine, one elevator, one dustbin, one door, and one heater
• the description of three areas: a seat area, an entrance, and a corridor.

The second example is a real scene of metro station in Nuremberg which has been selected in the european project AVS-PV. This is an entrance of a metro station, eight equipments and two areas have been defined. The equipment are six turnstiles and two ticket vending machines. The areas are an entrance and a corridor.

Figure 4 shows an example of the complete description of a ticket vending machine for a metro station in Nuremberg.

  

Figure 4: Example of description of a ticket vending machine