Training an unconstrained 6 DOF biomimetic robotic eye with deep reinforcement learning.

Abstract

Understanding how the brain controls rapid eye movements, known as saccades, remains an open challenge. In this work, we develop a biologically plausible computational model of a biomimetic eye, with six degrees-of-freedom (DOFs), to explore the control signals that drive these rapid ocular movements, and further compare their movement characteristics with known human behaviour. We hypothesise that saccade generation is governed by optimising a total reward, penalising costs such as inaccuracy, duration, energy and tendon tension, and train a model-free deep reinforcement learning algorithm under this constraint. Our results show that the emerging control strategies replicate approximate human-like saccadic characteristics, including the nonlinear main sequence relationships, compliance with Listing's and Donders' Laws, straight oblique trajectories, normometric pulse-step-like controls, and the antagonistic pairing of extraocular muscles, without explicitly enforcing these behaviours. We further analysed the evolution of the different costs during learning, and the impact of noise on the resulting control strategies and corresponding motions.