Thèse : Commande Référencée Vision d'un Robot de Taille de Vigne

Résumé

L’agriculture est un secteur clé confronté à des défis majeurs tels que l’augmentation de la demande alimentaire, les pénuries de main-d’œuvre et la nécessité de pratiques durables. Le vieillissement de la main-d’œuvre, l’exode des jeunes vers les villes et le changement climatique placent ce secteur à un tournant, où la robotique agricole apparaît comme une solution prometteuse. Cette thèse vise à améliorer les performances des systèmes autonomes de taille de la vigne. Les approches actuelles souffrent d’un manque de contrôle basé sur des capteurs, entraînant de nombreux échecs de positionnement. Pour y remédier, un système à retour visuel est proposé afin d’atteindre précisément les poses de coupe, même en présence de mouvements. Après avoir montré les limites des méthodes classiques de vision, une approche innovante combinant l’alignement de nuages de points par ICP et l’asservissement visuel basé sur la position (PBVS) est développée et validée expérimentalement sur un robot Franka Emika. Le système est ensuite étendu par un planificateur de trajectoires en ligne fondé sur une commande prédictive non linéaire, garantissant des mouvements fluides et robustes face aux perturbations. Enfin, des méthodes récentes basées sur l’apprentissage profond sont exploitées pour permettre un asservissement visuel basé sur l’image (IBVS), à partir de caractéristiques visuelles extraites et appariées automatiquement. Des expériences sur vignes réelles confirment l’efficacité et la robustesse des approches proposées.

Abstract

Agriculture is a crucial global sector which faces unprecedented challenges, including rising food demand from a growing population, significant labor shortages, and the urgent need for sustainable practices. The convergence of an aging agricultural workforce, youth migration to urban areas, and climate change, places agriculture at a juncture. In response, agricultural robotics is emerging as an element of a possible solution, providing a means to automate laborious tasks and address these pressing issues. This thesis aims to improve the performance of autonomous vine pruning systems. A significant limitation in current approaches is the absence of sensor based control, which frequently results in a high incidence of task failures—where the end-effector fails to reach the designated pruning location. This research addresses this critical gap by proposing a visual feedback–driven system designed to accurately reach the cutting poses, even under vine or robot movement.

Given the intricate structure of vine branches and the cluttered nature of vineyard environ- ments, an investigation is first conducted on classical methods for extracting visual descriptors (SIFT, ORB, D-SIFT, etc.) and the feasibility of fitting geometrical primitives (lines, cylinders, etc.) to be subsequently used for control. The results highlight the difficulty of the task when relying on classical methods. Then, a novel and general approach to vision-based vine pruning is proposed. It combines Iterative Closest Point (ICP) point cloud alignment with position-based visual servoing (PBVS). Four relevant ICP variants are compared within PBVS in vine pruning scenarios: standard ICP, Levenberg–Marquardt ICP, Point-to-Plane ICP, and Symmetric ICP. A dedicated ICP initial guess is incorporated to improve alignment speed and accuracy, as well as a procedure for generating reference point clouds at pruning locations. Live experiments conducted on a Franka Emika manipulator equipped with a stereo camera are reported, involving three real vines under laboratory conditions. Further, this system is augmented with an online point cloud based waypoint planner. Using a Nonlinear Model Predictive Control (NMPC) scheme, a sequence of 3D poses is computed in real time. Then, it is navigated by a sliding-reference PBVS ensuring smooth traversal without stopping. An ICP based point cloud alignment featuring a specific initial guess is incorporated for pose estimation. Qualitative and quantitative evaluations are conducted on a Franka Emika manipulator fitted with an eye-in-hand stereo camera, which performs pruning sequences on three increasingly complex vine stocks. The system robustness is validated against unexpected disturbances, including vine and robot base movement.

Last, recent advances in AI based visual processing are drawn upon, so as to overcome the difficulties in the extraction, matching and tracking of 2D visual features, and enable image-based visual servoing (IBVS). Deep learning (DL) based methods are used for feature extraction and matching. With this in mind, a dedicated dataset is built in order to train DL strategies to feature matching and vine branch segmentation. The IBVS algorithm is designed so as to ensure convergence despite the absence of depth information. Results of live experiments on real vines are provided, using a Franka Emika manipulator equipped with a single RGB camera. The work concludes with a discussion and directions for future research.