Interpreting reservoir computing through the equivalent visualization of its loss landscape.

Abstract

Reservoir computing (RC) has been recognized as a promising ultra-lightweight model, but its black-box nature renders its predictions less interpretable. As a result, interpreting the underlying prediction mechanism of RC is attracting increasing attention. Among the interpretability methods, visualization serves as an intuitive approach for interpreting RC, enabling even novices to directly observe RC loss landscape and its dependency on the parameter. All loss landscape visualizations are inevitably subject to information loss due to the low-dimensional projection constraints, yet existing methods lack justification for the effectiveness of their visualization plots. It naturally raises concerns about the reliability and utility of such visualization plots. To address this issue, we propose an equivalent visualization method to depict the RC loss landscape in the low-dimensional 2D or 3D space while maintaining high confidence. Specifically, the number of parameter orderings is first introduced to quantify the representativeness of RC parameter candidates, as the parameter ordering is experimentally identified as a predominant factor influencing RC predictions. Then, the periodic interpolation approach is introduced to generate the RC parameter candidates, based on which the 2D or 3D visualization plots of RC loss landscape are constructed. Theoretically, the generated parameter candidates can form a symmetric group encompassing all parameter orderings, ensuring that the resulting 2D or 3D visualizations equivalently reflect the original loss landscape from the perspective of parameter orderings. On this basis, how the loss landscape of RC relates to its trainability and key components (e.g., the hyperparameter) is visually interpreted. Furthermore, the reliability of the obtained visualization explanations is substantiated by the inherent memory property of RC. The proposed visualization method is model-agnostic, allowing its extension to portray the loss landscape of ResNet-56.