First Cycle Evaluation
Second Cycle Evaluation
Video surveillance is an important application of computer vision. Many years of research and experimentation have led to the development of innovative commercial applications. Nevertheless, video sequence analysis and interpretation is still a very active research area.
In the meantime, the maturity of this technology favours the realisation of a comparative study between existing methods.
The ETISEO Project seeks to work out a new structure contributing to an increase in the evaluation of video scene understanding; with the active participation of industrialists and many research laboratories, such as French, European and international partners.
ETISEO focuses on the treatment and interpretation of videos involving pedestrians and (or) vehicles, indoors or outdoors, obtained from fixed cameras.
This project is part of the Techno-Vision evaluation network funded by the French ministry of defence and the French ministry of research.
The Project Process
ETISEO is a two-years project: the first year is dedicated to data acquisition, metrics and evaluation definition and validation, while the second year entain running the real evaluation. ETISEO began in the first trimester in 2005, running through the end of 2006.
ETISEO consists in two evalation cycles:
- a first evaluation cycle from January to July 2006,
- a second cycle from June to December 2006.
First evaluation is intended to verify protocols, technical details as well as the pertinence of criteria and metrics. It is based on a specific data set (Video data corpus 1). A first comparison of algorithm performances is achieved at the end of this cycle. First evaluation starts in January and ends in July. Participants can submit their results until end of March, after that period we will analyse the results, the metrics, and the evaluation cycle.
Second evaluation is the real evaluation cycle. It will start in June. Participants can submit their results until end of September.
Third evaluation. We (at INRIA) continue to work on the evaluation to gain more insights into the problem. The ongoing research results will be pooled on the site too.
Three seminars during project life will allow partners and participants to share and collaborate with ETISEO:
- During first seminar meeting (May 2005) participants expressed their needs about metrics, datasets and rules.
- The second seminar (December 2005) was dedicated to the first evaluation launching.
- The third seminar was held at the end of the real evaluation cycle (December 2006).
Diffusion & dissemination
ETISEO dataset and evaluation tools should constitute a reference in "vehicles and pedestrians scene understanding".
Therefore, resources generated during the project: videos, ground-truth, and evaluation tools, is now made public and accessible on the Internet. Any laboratory should then be able to run its methods using the dataset and compare its results with those of the ETISEO evaluation.
© SILOGIC/INRIA 2007