Integrating Machine Learning with Musculoskeletal Simulation Improves OpenCap Video-Based Dynamics Estimation.

Abstract

Musculoskeletal dynamics influence the progression and rehabilitation of movement-related conditions. However, estimating whole-body dynamics using accessible tools, like smartphone video, remains challenging. Physics-based and machine learning (ML)-based dynamic predictions each offer advantages, but both approaches struggle to achieve high accuracy and physical realism. Here, we created a hybrid ML-simulation framework to improve estimates of ground reaction forces, joint moments, and joint contact forces from smartphone video kinematics. We used ML models to predict ground forces and centers of pressure from video-based kinematics. The hybrid framework generates a dynamic simulation that tracks predicted forces and kinematics while encouraging dynamic consistency. We compared the hybrid model with kinematic-tracking simulations and with ML-predicted forces applied via inverse dynamics. Performance was evaluated using mean absolute error relative to lab-based inverse dynamics using marker and force plate data from 10 individuals walking. The ML and hybrid approaches reduced vertical ground force error by 40-44% compared to simulation. The hybrid model improved joint moment accuracy by 29-45% and joint contact force accuracy by 12-13% compared to simulation- or ML-only approaches, with the largest improvements in peak medial knee contact force (49%) and knee adduction moment impulse (30%). Our hybrid model improves the accuracy of dynamics from smartphone videos during walking, outperforming simulation for ground forces and both simulation- and ML-only approaches for joint moments and contact forces. These methods enable more accurate, scalable assessments of musculoskeletal dynamics, supporting out-of-lab studies and precision treatment of gait-related conditions.