Security Versus Accuracy: Trade-Off Data Modeling to Safe Fault Classification Systems.

Abstract

While the data-driven fault classification systems have achieved great success and been widely deployed, machine-learning-based models have recently been shown to be unsafe and vulnerable to tiny perturbations, i.e., adversarial attack. For the safety-critical industrial scenarios, the adversarial security (i.e., adversarial robustness) of the fault system should be taken into serious consideration. However, security and accuracy are intrinsically conflicting, which is a trade-off issue. In this article, we first study this new trade-off issue in the design of fault classification models and solve it from a brand new view, hyperparameter optimization (HPO). Meanwhile, to reduce the computational expense of HPO, we propose a new multiobjective (MO), multifidelity (MF) Bayesian optimization (BO) algorithm, MMTPE. The proposed algorithm is evaluated on safety-critical industrial datasets with the mainstream machine learning (ML) models. The results show that the following hold: 1) MMTPE is superior to other advanced optimization algorithms in both efficiency and performance and 2) fault classification models with optimized hyperparameters are competitive with advanced adversarially defensive methods. Moreover, insights into the model security are given, including the model intrinsic security properties and the correlations between hyperparameters and security.