The aim is to develop “back-ends” that, based on “front-ends” performing low-level perceptual functions (detection, segmentation, temporal tracking, recognition, etc.), enable higher-level inferences: action recognition, behavior learning and prediction, localization and SLAM, etc. The underlying techniques relate to deep generative models, probabilistic graphical models and inference, causal graphical models, etc.

Vision-based recognition of human actions

In an “everyday” indoor environment, a human and objects with which he/she interacts are perceived using a 3D sensor. A first Bayes detector of actions combines 3D perception with prior knowledge (affordances, action transition probabilities, interaction modeling...) in order to infer posterior probabilities of action sequences. In parallel, spatio-temporal relationships are captured in a C3D-GRU recurrent neural network. These probabilistic and “data-driven” predictions are merged into a dense neural layer. The whole system is evaluated on several datasets: CAD-120, Watch-n-Patch, etc.

Doctoral thesis of Camille Maurice (Funding LAAS-CNRS/RAP). Contact: Frédéric Lerasle.

Short-term modeling and prediction of trajectories of interacting agents

A fleet of vehicles in a highway environment is connected to a centralized system in the Cloud. Each vehicle broadcasts its identifier, its absolute position as well as the statistics of the associated uncertainty. The centralized prediction of the agents' trajectories in the short term (typically 3-5 seconds) is carried out in order to characterize the collision risk. The challenge is to capture short-term and long-term interactions, and to get an explainable scheme, featuring good computational properties and allowing the management of false measurements. The solution consists of: the offline (supervised or unsupervised) learning of the ego-vehicle prior dynamics (conditioned on its past state and the state of its neighbors); the online inference by approximate Bayes filtering techniques, suited to deep dynamic models and very high-dimensional state spaces.

Doctoral thesis of Tristan Klempka (CIFRE with Continental Digital Services France). Contact: Patrick Danès.

Following this action, the analysis and prediction of the movements of heterogeneous agents (cars, bicycles, pedestrians, etc.) will be addressed in aerial images captured in urban scenarios.

Doctoral thesis starting in December 2024 (Joint funding of the Key Challenge on Intelligent and Sustainable Mobility in Occitanie (MIDOC, The Occitanie Region via the University of Toulouse), CEREMA and LAAS-CNRS/RAP. Co-supervision with CEREMA Toulouse). Contact: Patrick Danès.

SLAM

Using modern causal inference tools, in particular causal graphs and the structural causal model (SCM) promoted by Pearl and co-authors, an alternative view of the Simultaneous Localization and Mapping (SLAM) problem is developed to address some long-standing problems. The CARLIT (Causal Approach to Represent Locomotion and Internalize Topography) system articulates causal hypotheses and appropriately exploits conditional independencies for the treatment of loop closures or the incorporation of landmarks. The adjustment problem boils down to a constrained least squares optimization problem. A connection is made with the work of pioneers of probability theory such as Laplace, who handled uncertainty in a “quasi-deterministic” framework. Their emblematic application to the geodesic adjustment problem (triangulations for cartography) shares many structural commonalities with SLAM.

Doctoral thesis of Joël Tari (CIFRE AKKODIS company). Warm thanks to the autOCampus platform of the University Toulouse III Paul Sabatier for providing the framework for the experiments, to the company Soben/TwinswHeel for lending one of their robots, to the company GUIDE-GNSS for providing GNSS-RTK data with ground truth (via a very high-quality hybridation system).

As part of the NINSAR project funded by the PEPR Agroecology and ICT, the Doctoral thesis of Mathieu Fons has begun in September 2024 on the topic of cooperative and active SLAM (Co-supervision with IP Clermont-Ferrand and XLIM Limoges).

Learning of prior dynamics of errors of localization algorithms for their virtual emulation

The objective is to synthesize, on the basis of “datasets”, deep generative models of localization errors relative to a ground truth in various contexts relevant to agricultural robotics (e.g., for GNSS, open environment, canopy, proximity of metal structures, casual passage under buildings, etc.). Unsupervised or semi-supervised methods are sought. These generative models will be used in a virtual environment for the testing and certification of multi-sensor geopositioning systems.

France 2030 BPI GEOSUR Project. Collaboration with AgreenCulture (leader), INRAE, GUIDE-GNSS, M3 Systems, TORUS ACTIONS and the TRUST team of LAAS-CNRS.