Deep RL-Based Continuous-Time Continuous-Space Homeostatically Regulated Reinforcement Learning (CTCS-HRRL).

Abstract

Homeostasis is a biological process by which living organisms actively regulate internal states to maintain equilibrium. Prior research suggests that organisms can learn behaviors that support such regulation. The recently proposed homeostatically regulated reinforcement learning (HRRL) framework explains learned homeostatic regulation by integrating drive reduction theory with reinforcement learning (RL). While HRRL has been validated in discrete time and space, it has not yet been extended to continuous domains. In this work, we extend the HRRL framework to continuous time and space and introduce the continuous-time continuous-space HRRL (CTCS-HRRL) framework. We validate this framework using a computational model that simulates homeostatic regulation in biological agents. The model is grounded in the Hamilton-Jacobi-Bellman (HJB) equation and employs neural network-based function approximation within an RL setting. Through three simulation-based experiments, we demonstrate that the agent learns to dynamically select policies that maintain homeostasis under continuously changing internal states and external environments. These results establish CTCS-HRRL as a promising framework for modeling animal dynamics and decision-making in continuous domains.