Recent work, carried out in collaboration with the RIS research group at LAAS, has addressed the testing of a fault-tolerant temporal planner. The test objective was to verify to what extent the planner could indeed tolerate the faults it was designed to tolerate, i.e., design faults in its declarative domain model and search heuristics. We focused on faults in the domain model. To this end, we designed and implemented a fault-injection test environment that allowed the domain models to be mutated, using the SESAME mutation tool developed at LAAS [1]. The test environment comprises an open source robot simulator named Gazebo, an interface Library named Pocosim, and the components of the LAAS 3-layer architecture for autonomous systems (see figure). The control software was the exact same software used on real robots. However, a simulated robot was used in order to automate the testing activity (and to prevent the evident risks that would arise if faults were to be injected into software controlling a real robot). The outcome of each test was judged via the subset of goals that were successfully completed by the robot and performance results such as the mission execution time and the distance covered by the robot to reach its goals. The tests were conclusive in showing that the proposed mechanisms usefully improve the system behavior in the presence of model faults and do not severely degrade the system performance [2].

Fig 1: Testing of a fault-tolerant temporal planner by model mutation

Currently work is aimed at assessing the robustness of the functional (or reactive) layer of the LAAS 3-layer architecture for autonomous systems. This layer consists of a set of asynchronous modules that interact in real-time with the robot hardware (e.g., laser direction scanners, wheel motors, cameras, etc.). Robustness of the functional layer relates to its ability to defend itself against errors in, and multiple inter-leavings of, requests issued by client software at the next upper layer of the architecture (the OpenPRS procedural executive, cf. figure). In particular, we aim to assess the robustness of defenses automatically synthesized by Verimag’s BIP framework . The test environment under construction is, like in our earlier work, based on robot simulation and fault injection. Here, the faults are injected within client software within the procedural executive layer, with the aim of simulating erroneous client behavior. Test outcomes will be judged through observations of return codes from the functional layer, system status flags, values of physical variables of the robot simulator, and by comparison with execution traces in the absence of injected faults. This work is carried out in the framework of the MARAE project.

Future directions in this area include: a) the combination of intensive simulation testing and analytical techniques (such as model-checking); b) massive parallelization of the simulation testing activity, using a grid-based environment.

Key publications

[1] Y. Crouzet, H. Waeselynck, B. Lussier, D. Powell. “The SESAME Experience: from Assembly Languages to Declarative Models.” In Mutation 2006 - The Second Workshop on Mutation Analysis, 17th IEEE Int. Symp. on Software Reliability Engineering (ISSRE 2006), Raleigh, NC, USA, 2006 (DOI: 10.1109/MUTATION.2006.14, 10 pages).

[2] B. Lussier, M. Gallien, J. Guiochet, F. Ingrand, M.-O. Killijian, D. Powell. “Fault Tolerant Planning for Critical Robots.” In 37th Annual IEEE/IFIP Int. Conf. on Dependable Systems and Networks (DSN 2007), pp. 144-53, Edinburgh, UK, 2007.