Adversarial rain attack and defensive deraining for DNN perception.

Abstract

Rain often poses inevitable threats to deep neural network (DNN)-based perception systems, and a comprehensive investigation of the potential risks of rain to DNNs is of great importance. However, it is rather difficult to collect or synthesize rainy images that represent all rain situations that may occur in the real world. To this end, in this paper, we start from a new perspective and propose to combine two totally different studies, i.elet@tokeneonedot, rainy image synthesis and adversarial attack. We first present an adversarial rain attack, with which we could simulate various rain situations with the guidance of deployed DNNs and reveal the potential threat factors that can be brought by rain. In particular, we design a factor-aware rain generation that synthesizes rain streaks according to the camera exposure process and models learnable rain factors for adversarial attacks. With this generator, we conduct adversarial rain attacks against image classification and object detection. To defend the DNNs from the negative rain effect, we also present a defensive deraining strategy, for which we design an adversarial rain augmentation that uses mixed adversarial rain layers to enhance deraining models for downstream DNN perception. Our large-scale evaluation on various datasets demonstrates that our synthesized rainy images with realistic appearances not only exhibit strong adversarial capability against DNNs, but also boost the deraining models for defensive purposes, building the foundation for further rain-robust perception studies.