Characterization of Human Balance through a Reinforcement Learning-based Muscle Controller.

Abstract

Objective characterization of human balance remains a challenge and clinical observation-based balance tests during physical rehabilitation are often affected by subjectivity. On the other hand, computational approaches mostly rely on center of pressure (COP) tracking and inverted pendulum models, which do not capture the multi-joint and muscle contributions to whole-body balance. This study proposes a novel musculoskeletal modeling and control methodology to investigate human balancing capabilities in the center of mass (COM) state space. A musculoskeletal model is integrated with a balance controller trained through reinforcement learning (RL) to explore the limits of dynamic balance during postural sway. The RL framework consists of two interlinked neural networks (balance recovery and muscle coordination) and is trained using Proximal Policy Optimization (PPO) under multiple training strategies. By exploring recovery from random initial COM states with a trained controller, a balance region (BR) is obtained that encloses successful state-space trajectories. Comparing BRs obtained from different trained controllers with the analytical postural stability limits of a linear inverted pendulum model, we observe a similar trend in COM balanced states, but reduced recoverable areas. Furthermore, the effects of muscle weakness and neural excitation delay on the BRs are investigated, revealing reduced balancing capability in the COM state space. The novel approach of determining regions of stability through learning muscular balance controllers provides a promising avenue for personalized balance assessments and objective quantification of balance capability in humans with different health conditions.