Pl@ntNet Platform and Apps

Pl@ntNet is a participatory platform and information system dedicated to the production of botanical data through image-based plant identification. It is developed by scientists from four French research organisations (INRIA, Cirad, INRA, IRD) and Tela Botanica NGO with the financial support of Agropolis fondation. As the research supervisor of the platform, I did play a central role in the conception of the data models, system architecture and softwares. The mobile front-end of the platform has been downloaded by more than 3M persons in 170 countries, which makes it the Inria software with the largest audience. The platform generates each year millions of (noisy) plant observations that begin to be studied by researchers in ecology and (agro-)biodiversity. Pl@ntNet is on Facebook and Twitter.

Plantnet iphone and android app: an image sharing and retrieval application for the identification of plants. Among other features, this free app helps identifying plant species from photographs, through convolutional neural network technologies coupled with a visual search engine using several of my works on content-based image retrieval (e.g. A posteriori multi-probe and RMMH).

The Plant Game: a gamified application dedicated to the crowdsourced identification of plants (more than 25K users). Its objective is two folds: (i) train people to identify plants and (ii), validate/correct noisy plant observations. The software relies on several innovations related to the active training of the users, the tasks assignment and the Bayesian inference in the presence of highly partial knowledge. The software as well as the underlying new techniques have been entirely developed by Maximilien Servajean, a post-doc that I supervised jointly with Dennis Shasha (New-York univ.).

Smart'Flore android app for the discovery of the surrounding vegetal biodiversity. It includes three main features: (i) the geo-based exploration of the world’s largest repository of biodiversity occurrences (GBIF), (ii) the exploration of virtual botanical trails (created offline through a dedicated web application hosted by TelaBotanica NGO) and (iii) the access to a variety of information about the plants. Nowadays, it has been downloaded by more than 22K users. This work is part of the Floris'Tic project, supported by the "Programme Investissement d'Avenir" and involving teams of Agropolis Fondation, Tela Botanica, Inria, Cirad, Cnrs, Inra, Ird, UM.

Other softwares

• PMH: Probabilistic Multi-probe Hashing library - integrates most of my research works on hashing and approximate k-nn search, as well as other methods of the state-of-the-art. One of its main advantage, besides scalability, is its genericity in terms of query types, metrics and data formats. It is currently running on the server side of the Pl@ntNet application, as well as in the DigInPix web application (developed by INA). It is integrated in the SnoopIm and IKONA softwares.
• SnoopIm: Geometrically Consistent Retrieval and Mining of Visual Entities - integrates most of my works on visual objects retrieval as well as the work of two of my PhD students (Pierre Letessier and Valentin Leveau). It is the core technology of the DigInPix web application (developed by INA) and it is currently being integrated in the CetaNet web platform (aimed at monitoring whales in the context of a collaboration with two NGO’s: CetaMada and Globis) as well as an experimental platform dedicated to Egyptian hieroglyphs.
• INA Video Copy Detection software - used in INA SIGNATURE product by INA, one of the world leader in digital archiving. This system continuously monitors hundreds of TV channels in order to automatically detect the broadcast of video sequences belonging to a large reference catalog (with more than 100,000 hours of archives). The technology is used by several major French actors in the TV, film and web industry (TF1, Canal+, M6, Dailymotion, TDF, EuropaCorp). My contribution concerns the development of the core content-based video retrieval technology.
• IKONA (CBIR engine) - IKONA is a generalist software dedicated to content-based visual information indexing and retrieval. Its main functionalities are the extraction, the management and the indexing of visual features. It offers a wide range of interactive search and navigation methods including query-by-example, query-by-window, relevance feedback or search results clustering. I have been actively participating to the conception and development of the core engine. It is currently running on the server side of the Pl@ntNet application.

Patents

• US patent US2006083429, "Search of similar features representing objects in a large reference", A. Joly, patent deposited by INA, April 2006.
• European patent EP1650683, "Search of similar features representing objects in a large reference", A. Joly, patent deposited by INA, April 2006.