Main

A deep learning approach to Species Distribution Modelling

[Author version] [Springer book]
Species distribution models (SDM) are widely used for ecological research and conservation purposes. The environment is in most cases represented by climate data (such as temperature, and precipitation) and other variables such as soil type or land cover can also be used. In this paper, we propose a deep learning approach to the problem in order to improve the predictive effectiveness. Non-linear prediction models have been of interest for SDM for more than a decade but our study is the first one bringing empirical evidence that deep, convolutional and multilabel models might participate to resolve the limitations of SDM. Indeed, the main challenge is that the realized ecological niche is often very different from the theoretical fundamental niche, due to environment perturbation history, species propagation constraints and biotic interactions. Thus, the realized abundance in the environmental feature space can have a very irregular shape that can be difficult to capture with classical models. Deep neural networks on the other side, have been shown to be able to learn complex non-linear transformations in a wide variety of domains. Moreover, spatial patterns in environmental variables often contains useful information for species distribution but are usually not considered in classical models. Our study shows empirically how convolutional neural networks efficiently use this information and improve prediction performance.

Multimedia Tools and Applications for Environmental & Biodiversity Informatics

[Author version] [Springer book]

This edited volume focuses on the latest and most impactful advancements of multimedia data globally available for environmental and earth biodiversity. The data reflects the status, behavior, change as well as human interests and concerns which are increasingly crucial for understanding environmental issues and phenomena. This volume addresses the need for the development of advanced methods, techniques and tools for collecting, managing, analyzing, understanding and modeling environmental & biodiversity data, including the automated or collaborative species identification, the species distribution modeling and their environment, such as the air quality or the bio-acoustic monitoring. Researchers and practitioners in multimedia and environmental topics will find the chapters essential to their continued studies.

Crowdsourcing Thousands of Specialized Labels: a Bayesian active training approach

[Author version] [Transanctions on Multimedia]
In classical crowdsourcing frameworks, the labels correspond to well known or easy-to-learn concepts so that it is straightforward to train the annotators by giving a few examples with known answers. Neither is true when there are thousands of complex domain-specific labels. The originality of this work is to focus on such annotations that usually require hard expert knowledge (such as plant species names, architectural styles, medical diagnostic tags, etc.). We consider that common knowledge is not sufficient to perform the task but any people can be taught to recognize a small subset of domain-specific concepts. In such a context, it is best to take advantage of the various capabilities of each annotator through teaching (annotators can enhance their knowledge), assignment (annotators can be focused on tasks they have the knowledge to complete) and inference (different annotator propositions can be aggregated to enhance labeling quality). In this work, is a set of theoretical contributions and data-driven algorithms to allow the crowdsourcing of thousands of specialized labels thanks to the pro-active training of the annotators. The framework relies on deep learning, variational Bayesian inference and task assignment to adapt to the skills of each annotator both in the questions asked and the weights given to their answers. The underlying judgements are Bayesian, based on adaptive priors. To achieve live experiments, the whole framework has been implemented as a serious game available on the web (ThePlantGame).

Large-scale Content-based Visual Information Retrieval

[HDRThesis2015] HDR habilitation (highest French academic qualification) - defended the 26/05/2015

Rather than restricting search to the use of metadata, content-based information retrieval methods attempt to index, search and browse digital objects by means of signatures or features describing their actual content. This thesis describes several of my works related to this domain. The different contributions are presented in a bottom-up fashion reflecting a typical three-tier software architecture of an end-to-end multimedia information retrieval system. The lowest layer is only concerned with managing, indexing and searching large sets of high-dimensional feature vectors. The middle layer rather works at the document level and is in charge of analyzing, indexing and searching collections of documents. The upper layer works at the applicative level and is in charge of providing useful and interactive functionalities to the end-user.

Kernelizing Spatially Consistent Visual Matches

[ICMR2015]
This paper introduces a new image representation relying on the spatial pooling of geometrically consistent visual matches. We therefore introduce a new match kernel based on the inverse rank of the shared nearest neighbors combined with local geometric constraints. To avoid overfitting and reduce processing costs, the dimensionality of the resulting over-complete representation is further reduced by hierarchically pooling the raw consistent matches according to their spatial position in the training images. The final image representation is obtained by concatenating the resulting feature vectors at several resolutions. Learning from these representations using a logistic regression classifier is shown to provide excellent fine-grained classification performances outperforming the results reported in the literature on several classification tasks.

Interactive plant identification based on social images

[EcologicalInformatics2014]
Initiated in the context of a citizen sciences project, the main contribution of this work is an innovative collaborative workflow focused on image-based plants identi cation as a mean to enlist new contributors and facilitate access to botanical data. Since 2010, hundreds of thousands of geo-tagged and dated plant photographs were collected and revised by hundreds of novice, amateur and expert botanists of a specialized social network. An image-based identi cation tool - available as both a web and an iPhone application - is synchronized with that growing data and allows any user to query or enrich the system with new observations. An important originality is that it works with up to five di fferent organs (pdf) contrarily to previous approaches that mainly relied on the leaf. This allows querying the system at any period of the year and with complementary images composing a plant observation. Extensive experiments of the visual search engine show that it is already very helpful to determine a plant among hundreds or thousands of species (to appear). At the time of writing, the whole framework covers about half of the plant species living in France (3500 species), which already makes it the widest existing automated identi cation tool.

The data collected through this workflow is used each year in the LifeCLEF evaluation campaign that I am coordinating since 2011:

Scalable Mining of Small Visual Objects

[ACM-MM2012]
Automatically linking multimedia documents that contain one or several instances of the same visual object has many applications including: salient events detection, relevant patterns discovery in scientific data or simply web browsing through hyper-visual links. In this work pdf, we formally revisited the problem of mining or discovering such objects, and introduced a new hashing strategy, working first at the visual level, and then at the geometric level. Experiments conducted both on FlickrBelgaLogo dataset and on millions of images shows the efficiency of our method. Applying this technique to web images allows to suggest trustful hyper-visual links to the user and finally allows him to browse the web in a radically new way as illustrated in this video:

This new search paradigm is published in MTAP-2013 and will be demonstrated at ACM MM 2013.

Visual based Event Mining

[ICMR2011] [ACM-MM2012]
A Phd student of mine (Riadh Trad) did work on visual-based event retrieval and discovery in social data (Flickr images). He built a new event records matching technique making use of both the visual content and the social context pdf.

Besides, our objects mining and retrieval techniques were integrated within a visual-based media event detection system in the scope of a French project called the transmedia observatory. It allows the automatic discovery of the most circulated images across the main news media (news websites, press agencies, TV news and newspapers). The main originality of the detection is to rely on the transmedia contextual information to denoise the raw visual detections and consequently focus on the most salient trans-media events. This work was presented at ACM Multimedia Grand Challenge 2012 and obtained a grant. The movie presented during this event is available here:

Hash-based SVM approximation

[BMVC2012]
We addressed the problem of speeding-up the prediction phase of linear Support Vector Machines via Locality Sensitive Hashing pdf. Whereas the mainstream work in the field is focused on training classifiers on huge amount of data, less efforts are spent on the counterpart scalability issue: how to apply big trained models efficiently on huge non annotated collections ? In this work, we propose building space-and-time-efficient hash-based classifiers that are applied in a first stage in order to approximate the exact results and filter the hypothesis space. Experiments performed with millions of one-against-one classifiers show that the proposed hash-based classifier can be more than two orders of magnitude faster than the exact classifier with minor losses in quality.

Random Maximum Margin Hashing

[CVPR2011]
RMMH is a new hashing function aimed at embedding high dimensional feature spaces in compact and indexable hash codes. Several data dependent hash functions have been proposed recently to closely fit data distribution and provide better selectivity than usual random projections such as LSH. However, improvements occur only for relatively small hash code sizes up to 64 or 128 bits. As discussed in the paper, this is mainly due to the lack of independence between the produced hash functions. RMMH attempts to solve this issue in any kernel space. Rather than boosting the collision probability of close points, our method focus on data scattering. By training purely random splits of the data, regardless the closeness of the training samples, it is indeed possible to generate consistently more independent hash functions. On the other side, the use of large margin classifiers allows to maintain good generalization performances. Experiments show that our new Random Maximum Margin Hashing scheme (RMMH) outperforms four state-of-the-art hashing methods, notably in kernel spaces.

We recently investigated the use of RMMH for efficiently approximating K-NN graphs pdf, particularly in distributed environments. We highlighted the importance of balancing issues on the performance of such approaches and show why the baseline approach using Locality Sensitive Hashing does not perform well.

Logo retrieval with a contrario visual query expansion

I did work on logo retrieval within VITALAS European project as an application of large scale local features matching. I introduced a new visual query expansion method using an a contrario thresholding strategy in order to improve the accuracy of expanded query images [ACM09logo]. I also created a new challenging dataset, called BelgaLogos, which was created in collaboration with professionals of a press agency, in order to evaluate logo retrieval technologies in real-world scenarios.

Take a look to the flash demo:

Visit BelgaLogos home page to get the evaluation dataset

Interactive objects retrieval with efficient boosting

We developed jointly with my Phd student Saloua Litayem an efficient boosting method [ACM09boosting] to predict local feature based trained classifiers in sublinear time. This technique allows online relevance feedback or active learning on image regions.

High-dimensional Hashing

We developed, jointly with Olivier Buisson at INA, a new similarity search structure [ACM08AMP-LSH] dedicated to high dimensional features. It is built on the well-known LSH technique, but in order to reduce memory usage, it intelligently probes multiple buckets that are likely to contain query results in a hash table. This method is somehow inspired by our previous works on space-filling curve based hashing (see this one or this one for example) and improves upon recent theoretical work on multi-probe and query adaptive LSH. We hope providing an open source version in the next few months...

Content-based Video Copy Detection

The last research paper somehow summarizing my work on this topic: [TMA07]

My My Phd at INA is hopefully also still of interest ;-)

I am currently less involved on research issues about CBCD but still strongly implied in benchmarking (more info here).

I have also been leading a "show-case" on content-based copy detection with the objective to present user-oriented demos in professional and scientific meetings (more info here).

You can watch THE MOVIE :

Visual Local Features

  • Constant tangential angle interest points [CIVR07]

We developped with my master student, Ahmed Rebai, a new symmetry oriented interest point detector based on gradient orientations convergence. The aim was to reach better visual saliency than current detectors, such as Harris or Difference of Gaussians points. The developed technique gives very promising object class recognition performances.

Dissociated dipoles [DIPOLES07]

I have also been working on new local descriptors dedicated to transformed images retrieval, with a small dimension (20) and a high discrimination power. These descriptors obtained the best results in ImagEval benchmark both in term of recall/precision and search rapidity.

Density-based selection of local features

This work started in collaboration with the NII (National Institute of Japan) within the scope of my visit in Tokyo (july 2005). As common local features are selected only according to the local information content in the image, there is no guaranty that they will be distinctive in a large set of images. To overcome this issue, I introduced a new method [MIR05] to select relevant local features directly according to their discrimination power in a specific set of images. By computing the density of the local features in a source database with an efficient non parametric density estimation technique, it is indeed possible to select quickly the most rare local features in a large set of images.

left: 20 most salient points - right: 20 most rare points

Research

Personal

* [[Personal/Photo Page | Photos]]

edit SideBar

Blix theme adapted by David Gilbert, powered by PmWiki