Participants: Francois Bremond, Alberto Avanzi, Frederic Cupillard, Monique Thonnat, Christophe Tornieri.
In March 2003, at the end of the European Project ADVISOR, evaluation, validation and demonstration of the prototype have been done at the TMB headquarter in Barcelona to various guests, including the European Commission, project Reviewers and representatives of Bruxelles and Barcelona Metro. Together with this demonstration, an evaluation has been done by security operators of the metro of Barcelona and Brussels in charge of the videosurveillance during a week at the Sagrada Familia Metro Station in Barcelona.
|
The evaluation, validation and demonstration were conducted using both live and recorded videos. For the validation task, the system was tested in live conditions using four input channels in parallel, the four channels being composed of three recorded sequences and one live input stream from the main hall of the Sagrada Familia Metro station. The three recorded sequences enabled to test the system with rare scenarios of interest, not always available during the demonstration. The three recorded data sequences were constructed using thirty-two shorter prerecorded sequences, showing five different predefined scenarios, four of them (fighting, blocking, jumping over the barrier and vandalism) played by actors and one (overcrowding) coming from original videos. The live camera allowed to evaluate the system against scenarios which often happen (e.g. overcrowding) and which can occur during the demonstration. It also allowed to evaluate the system against false alarms.
|
In total, out of 21 fighting incidents in all the Demonstrator sequences, 20 alarms were correctly generated, giving a very good detection rate of 95%. These twenty correctly identified alarms had an average report accuracy of 68% (by accuracy we mean the temporal overlap between intervals corresponding to the detected alarm and the ground truth). Out of nine blocking incidents, seven alarms were generated, giving a detection rate of 78%. These seven alarms were found to be 60% accurate on average. Out of 42 instances of jumping over the barrier, including repeated incidents, the behaviour was detected 37 times, giving a success rate of 88%. The two sequences of vandalism were always detected with an overall accuracy of 71%, over six instances of vandalism. Finally, the two overcrowding alarms in camera C11 were consistently detected, with an overall accuracy of 80% over 7 separate instances of the alarms. The overcrowding alarms were also consistently detected in the live camera C10, with some 28 separate events being detected.
In conclusion, the ADVISOR demonstration has been evaluated very positively by end-users and European Committee. The algorithms responded very successfully to the input data, with high detection rates, less then 5% of false alarms and with all the reports being above approximately 70% accurate. A documentary movie of 11 minutes has been realised to present the ADVISOR system and to show the live demonstration in Barcelona. This work has been reported in several papers.
|
For more information, see the European Project ADVISOR Web-Site - Annotated Digital Video for Intelligent Surveillance and Optimised Retrieval -