A tree-structure-guided graph convolutional network with contrastive learning for the assessment of parkinsonian hand movements.

Abstract

Bradykinesia is one of the core motor symptoms of Parkinson's disease (PD). Neurologists typically perform face-to-face bradykinesia assessment in PD patients according to the Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS). As this human-expert assessment lacks objectivity and consistency, an automated and objective assessment scheme for bradykinesia is critically needed. In this paper, we propose a tree-structure-guided graph convolutional network with contrastive learning scheme to solve the challenge of difficulty in fine-grained feature extraction and insufficient model stability, finally achieving the video-based automated assessment of Parkinsonian hand movements, which represent a vital MDS-UPDRS component for examining upper-limb bradykinesia. Specifically, a tri-directional skeleton tree scheme is proposed to achieve effective fine-grained modeling of spatial hand dependencies. In this scheme, hand skeletons are extracted from videos, and then the spatial structures of these skeletons are constructed through depth-first tree traversal. Afterwards, a tree max-pooling module is employed to establish remote exchange between outer and inner nodes, hierarchically gather the most salient motion features, and hence achieve fine-grained mining. Finally, a group-sparsity-induced momentum contrast is also developed to learn similar motion patterns under different interference through contrastive learning. This can promote stable learning of discriminative spatial-temporal features with invariant motion semantics. Comprehensive experiments on a large clinical video dataset reveal that our method achieves competitive results, and outperforms other sensor-based and RGB-depth methods. The proposed method leads to accurate assessment of PD bradykinesia through videos collected by low-cost consumer cameras of limited capabilities. Hence, our work provides a convenient tool for PD telemedicine applications with modest hardware requirements.