论文标题
神经网络控制系统的安全验证
Safety Verification of Neural Network Controlled Systems
论文作者
论文摘要
在本文中,我们提出了一种系统级别的方法,用于验证神经网络控制系统的安全性,将连续时间的物理系统与基于离散的神经网络控制器相结合。我们假设控制器的通用模型可以捕获涉及神经网络的简单和复杂行为。基于此模型,我们执行可及性分析,该分析近似于整体系统的可触及状态,从而实现了正式的安全证明。为此,我们利用了验证的模拟来近似物理系统的行为和抽象的解释,以近似控制器的行为。我们使用现实世界中的用例来评估方法的适用性。此外,我们表明,当无法证明完全安全时,我们的方法可以提供有价值的信息。
In this paper, we propose a system-level approach for verifying the safety of neural network controlled systems, combining a continuous-time physical system with a discrete-time neural network based controller. We assume a generic model for the controller that can capture both simple and complex behaviours involving neural networks. Based on this model, we perform a reachability analysis that soundly approximates the reachable states of the overall system, allowing to achieve a formal proof of safety. To this end, we leverage both validated simulation to approximate the behaviour of the physical system and abstract interpretation to approximate the behaviour of the controller. We evaluate the applicability of our approach using a real-world use case. Moreover, we show that our approach can provide valuable information when the system cannot be proved totally safe.