Improving human motion generation based on a head-mounted display and its controllers via noise-augmented motion data and the recurrent inference model.

Abstract

Previous full-body human motion generation methods based on a head-mounted display (HMD) and two hand-held controllers rely on the assumption that the connections between the devices and body parts are rigid. Such approaches are not robust to motion noises, which are frequently introduced during virtual reality (VR) experience as these devices are often loosely attached to body parts. To address this problem, we propose a new generative strategy that synthesizes noise-augmented motion data to enhance the performance of human motion generation models and compared it against other three training strategies, including the direct, fine-tuning, and additive strategies. Furthermore, we present the Recurrent Inference Model (RIM), an approach that further improves the performance of full-body human motion generation. Our RIM model trained on noise-augmented motion data demonstrates consistent improvements over other existing state-of-the-art (SOTA) human motion generation methods during offline and real-time evaluation. The code of this research is available at: https://github.com/vrlab561/RIM-release.