Defenses for autonomously-adapting systems

Autonomous systems have to cope with various execution environments while guaranteeing safety, and in particular when they interact with humans, as is the case for robotic systems. Such critical systems are difficult to validate due to their high complexity and the fact that they operate within complex, variable and uncertain environments in which it is difficult to predict all possible system behaviors. As a result, autonomous systems have to be equipped with means for context-dependent safety enforcement. We consider here a device called a safety monitor, which has access to the necessary means for context observation (i.e., sensors) and is able to trigger, when necessary, appropriate safety enforcement.

We have addressed the process for eliciting safety rules that will be checked by a safety monitor during operation, considering the system as a black box [MBGPR12]. The proposed process is based on safety constraints identified through a hazard analysis using HAZOP-UML. The constraints are formalized and analyzed to identify safety rules. We distinguish initiative rules and restriction rules [MBGPR13]. An initiative rule launches an action to change the state, when it has been detected that the system has entered a warning (or safety margin) state. On the contrary, a restriction rule inhibits certain state changes, e.g., by means of an interlock device or by request filtering. This systematic process is based on the use of graphs and formal logic to determine safety margins and possible rules.

System state conceptual model for safety monitoring

We have also investigated the case when the system can be considered as a white box. The safety requirements are defined in a safety monitor that can control the internal behavior of the system. Formally, the symbols of the automaton implementing the monitor (Â) are transitions of the automaton (A) defining the controlled system [CM09]. An operator composing A and  allows a safe behavior of the system to be obtained [CM10]. This approach facilitates the expression of complex safety rules. It can be used to implement a functional safety approach (IEC 61508) [CM11].

Publications

[MBGPW13] M. Machin, J-P Blanquart, J. Guiochet, D. Powell and H. Waeselynck, Specifying safety monitors for autonomous systems, In: Fast abstracts, 32nd International Conference on Computer Safety, Reliability and Security (SAFECOMP 2013), Toulouse, 2013

[MBGPR12] A. Mekki-Mokhtar, J.P. Blanquart, J. Guiochet, D. Powell, M.Roy: Safety trigger conditions for critical autonomous systems. In: The 18th IEEE Pacific Rim International Symposium on Dependable Computing (PRDC 2012), Niigata, Japan, 2012

[CM11] Z. Chen, G. Motet, Methodology and Experience for Designing Safety-Related Systems in IEC 61508, 4th International Conference on Dependability (DEPEND 2011), Nice, France, IARIA publisher (August 2011) 57-64

[CM10]  Z. Chen, G. Motet, Towards Better Support for the Evolution of Safety Requirements, ACM/IEEE 32nd International Conference on Software Engineering (ICSE 2010), Cap Town, South Africa, IEEE Computer Society publishers, vol. 2 (May 2010) 219-222

[CM09]  Z. Chen, G. Motet, System Safety Requirements As Control Structures, 33rd Annual IEEE International Computer Software and Applications Conference (COMPSAC 2009), Seattle, Washington, USA, IEEE Computer Society publishers (July 2009), Vol. 1, 324-331

Back to TSF Research Topics page