Accurate and Robust Neural Architecture Search via a Flexible Supernet.

Abstract

Neural architecture search (NAS) has been widely adopted to design high-accuracy architectures, which are often vulnerable against adversarial attacks. To address this problem, existing robust NAS methods mainly focus on optimizing both natural accuracy and adversarial robustness in a fixed supernet, which is designed for natural accuracy. As a result, the derived architectures have the same construction scheme as the supernet, suffering from limited adversarial robustness and flexibility. In this article, we present the ARNAS++ method to search for accurate and robust neural architectures via a supernet with flexible parameter budgets and width. Specifically, we propose a parameter budget controlling loss to make architectures contain less parameters in the rear cells, based on which the adversarial robustness can be guaranteed. Moreover, we also propose a learnable filter number reduction ratio to control the filter numbers in the supernet, which can find more robust architectures beyond the fixed supernet, and make the supernet more flexible at the same time. We conduct experiments on six widely used benchmark datasets against the state of the art. The experimental results demonstrate that the proposed ARNAS++ method outperforms the competitors in terms of both natural accuracy and adversarial robustness under various popular adversarial attacks. In addition, the ablation studies show the effectiveness of the designed components and their positive contributions to the overall performance. The source code is available at: https://github.com/fyqsama/ARNASpp.