Ontology-Based Event Detection for Knowledge Discovery

Participants: Francois Bremond, Etienne Corvee, Jose Luis Patino Vilchis, Monique Thonnat.

The CARETAKER (Content Analysis and REtrieval Technologies to Apply Extraction to massive Recording) project aims at studying, developing and assessing multimedia knowledge-based content analysis, knowledge extraction components and metadata management sub-systems in the context of automated situation awareness, diagnosis and decision support. More precisely, CARETAKER will focus on the extraction of structured knowledge from large multimedia collections recorded over networks of cameras and microphones deployed in real sites. The produced audio-visual streams, if stored and automatically analyzed, represent a useful source of information in urban/environment planning, resource optimization, disabled/elderly person monitoring, etc., in addition to surveillance and safety issues.

We have considered two types of content knowledge: a first layer of primitive events that can be extracted from the raw data streams, such as ambient sounds, the degree of crowding present in the scene and the routes taken by individual people. A second layer of higher semantic events is defined based on more complex relationships between the primitive events and detected from longer term analysis.

Real testbed sites inside the metro of Roma and Torino, involving more than 30 sensors each (20 cameras and 10 microphones), has been provided. Additionally, the identification of the real user needs and beneficial use-case scenarios has served as a reference point for the correct framing of the semantic description scheme i.e. the ontology, the knowledge extraction components and the interface and demonstrator optimization.

Figure 1. Illustrations of Event Detection.

Within CARETAKER, the Orion project team is in charge of the long term tracking of objects of interest, the ontology-based event detection and of knowledge discovery.

For more information, see the CARETAKER Project Web-Site