论文标题
朝着基于概率的安全验证机器学习的组件对系统的安全验证
Towards Probability-based Safety Verification of Systems with Components from Machine Learning
论文作者
论文摘要
机器学习（ML）最近创建了许多新的成功案例。因此，在软件密集型系统（包括安全至关重要的系统）中使用ML技术有很大的动力。这就提出了MLB的安全验证问题，该系统目前被认为是不可行的，或者至少非常困难。我们认为，它需要考虑ML技术的特定属性，例如：（i）大多数ML方法都是归纳的，这既是它们的力量，又是其错误源。 （ii）深度学习产生的神经网络（NN）处于目前的最新状态。因此，总会剩下错误，至少对于深入NN（DNN），对其内部结构的验证非常困难。通常，安全工程不能完全保证不会造成任何伤害。这就是为什么使用概率（例如指定风险或可容忍的危险率（THR）的原因）。在本视觉论文中，我们提出了基于通过受控实验估计的错误概率和电感学学习的分类器本身估计的验证。概括误差范围可能会传播到危险的概率，而危险的概率不得超过THR。结果，定量确定的结合了安全关键系统中ML组件的分类误差的可能性，这对后者的总体安全验证有了明确的方式。
Machine learning (ML) has recently created many new success stories. Hence, there is a strong motivation to use ML technology in software-intensive systems, including safety-critical systems. This raises the issue of safety verification of MLbased systems, which is currently thought to be infeasible or, at least, very hard. We think that it requires taking into account specific properties of ML technology such as: (i) Most ML approaches are inductive, which is both their power and their source of error. (ii) Neural networks (NN) resulting from deep learning are at the current state of the art not transparent. Consequently, there will always be errors remaining and, at least for deep NNs (DNNs), verification of their internal structure is extremely hard. In general, safety engineering cannot provide full guarantees that no harm will ever occur. That is why probabilities are used, e.g., for specifying a risk or a Tolerable Hazard Rate (THR). In this vision paper, we propose verification based on probabilities of errors both estimated by controlled experiments and output by the inductively learned classifier itself. Generalization error bounds may propagate to the probabilities of a hazard, which must not exceed a THR. As a result, the quantitatively determined bound on the probability of a classification error of an ML component in a safety-critical system contributes in a well-defined way to the latter's overall safety verification.
