论文标题
正常数据的子群体检测
Sub-clusters of Normal Data for Anomaly Detection
论文作者
论文摘要
数据分析中的异常检测是一个有趣但仍充满挑战的研究主题。随着数据维度的复杂性的增加，它需要了解其描述中的语义上下文，以进行有效的异常表征。但是，现有的异常检测方法显示出有限的性能，具有高维数据（例如Imagenet）。现有研究评估了它们在低维，清洁和良好分开的数据集（例如MNIST和CIFAR-10）上的性能。在本文中，我们研究了具有高维和复杂正常数据的异常检测。我们的观察结果是，一般而言，异常数据是由可解释的特征定义的，这些特征也可以用于定义正常数据的语义亚群体。我们假设，如果存在相当好的特征空间在语义上可以分离给定正常数据的子群体，则看不见的异常也可以在空间中与正常数据有很好的区分。我们建议在给定的正常数据上执行语义聚类，并训练分类器，以了解最终进行异常检测的区分特征空间。基于我们对MNIST，CIFAR-10和Imagenet进行仔细而广泛的实验评估，具有正常和异常数据的各种组合，我们表明我们的异常检测方案优于最先进的方法的状态，尤其是在高维真实世界图像的情况下。
Anomaly detection in data analysis is an interesting but still challenging research topic in real world applications. As the complexity of data dimension increases, it requires to understand the semantic contexts in its description for effective anomaly characterization. However, existing anomaly detection methods show limited performances with high dimensional data such as ImageNet. Existing studies have evaluated their performance on low dimensional, clean and well separated data set such as MNIST and CIFAR-10. In this paper, we study anomaly detection with high dimensional and complex normal data. Our observation is that, in general, anomaly data is defined by semantically explainable features which are able to be used in defining semantic sub-clusters of normal data as well. We hypothesize that if there exists reasonably good feature space semantically separating sub-clusters of given normal data, unseen anomaly also can be well distinguished in the space from the normal data. We propose to perform semantic clustering on given normal data and train a classifier to learn the discriminative feature space where anomaly detection is finally performed. Based on our careful and extensive experimental evaluations with MNIST, CIFAR-10, and ImageNet with various combinations of normal and anomaly data, we show that our anomaly detection scheme outperforms state of the art methods especially with high dimensional real world images.
