In this more prospective topic, RAP initiated in 2004, a new theme aiming at the design, development and evaluation of integrated sensors. In a long-term objective, a system is seen as a network of integrated and decentralized units, communicating to each other, high level and abstract information in order to achieve a common goal : this concept can be applied for a robot or for a monitoring system made of a network of smart cameras. During the last period, RAP has cooperated with other LAAS research group in order to study the hardware and software architecture of sensory systems able both to execute complex algorithms (dense stereovision, detection and tracking of humans, obstacle detection on vehicles), and to satisfy hard constraints like real time performance, compacity and autonomy (low power, wifi communication).

The design and the implementation of an integrated stereovision sensor, were studied in the context of the internal PICAS$O LAAS project [IAV2007]. The first version developed on an Altera evaluation system (Quartus, Stratix kit) provided 640x480 disparity images at 100Hz ; it is currently improved and  adapted to be executed on a compact processing unit (6x6cm cards with Cyclone2 FPGA, connected to cameras through CameraLink interfaces),  developed with the SME Delta Technologies Sud Ouest (DTSO).  See hereafter the architecture designed to integrate the stereovision algorithm :

• Three images could be processed : two acquired by CCD or CMOS cameras in the visible bandwidth for the stereovision algorithm, a third one could be acquired on the same processing unit, typically an FIR image in order to map aparent temperatures on a 3D shape.
  
• Two FPGA cards are devoted to the rectification functions, respectively applied to the left and right stereo images ;
  
• The third one is in charge of the correlation function, to provide the disparity map.
• Finally the fourth card could be used either to integrate filters on the disparity map or to fuse disparity and thermal maps (not studied by now).
  

 

Besides, research work has been initiated about the development of new algorithms on human detection and tracking from a network of integrated monitoring cameras developed also by DTSO, so based on the same FPGA cards. This research work began in the context of a project with University of Mondragon in Spain, and will continue with the project CameraNet funded by the Midi-Pyrénées region. The aim consists in the adaptation of functions developed in order to detect and track humans using particle filtering, then in the integration of these functions on the camera, and finally on the achievement of monitoring tasks by a wifi network of such cameras, exchanging information on the environment state.

Finally concerning obstacle detection, ongoing work tackles the design and the implementation of an obstacle detection system to be integrated on a robot or a vehicle. It is made of  N micro-cameras (N=8 for the first version) interfaced with a FPGA-based processing unit, now a StratixIII development kit. Three steps are required : at first the detection of  potential obstacles from a classification of every  pixel on every image as Ground or Obstacle, based on color-texture characteristics ; then a validation based on spatio-temporal analysis, using an estimation of the robot motion between successive acquisitions ; and finally, the fusion of all results in a probabilistic robot-centered occupancy grid, sliding with the robot during motions. This occupancy grid will be updated and sent to the robot control unit, at 30Hz [ICAR2009].
RAP will study how to integrate on a dedicated architecture, algorithms developed for obstacle detection from multi-spectral images acquired on an airplane moving on taxiways.

On-going PhDs (in cooperation with the N2IS research group at LAAS)

• W.Filali for the integration of human detection, tracking and identification functions, and for the use of a camera net.
  
• D.Botero, for the integration of obstacle detection function, from infrared cameras.
  
• M.Ibarra, on the integration of a microcameras belt, used for obstacle detection on a service robot.