Fluid Intake Action Detection Based on Egocentric Videos and YOLOv8 Models.

Abstract

Dehydration in older adults poses significant health risks, requiring effective monitoring solutions. This study addresses the challenge of detecting fluid intake accurately using a first-person, vision-based approach with wearable cameras and advanced object detection models. We developed a comprehensive dataset comprising 17 hours of drinking footage (∼3100 events) and 15 hours of non-drinking activities (∼3600 events) recorded as interference, from 36 participants, collected between October 2022 and January 2023 at King's College London. We include various container types and daily activities to enhance the model's robustness and generalizability. YOLOv8 models were used to detect drinking-related objects, and a mechanism was developed to analyse the size and position of the detection output to identify hand-container interactions and movements. The models achieved mAP@50 over 0.97 and F1-score over 0.95 in detecting drinking-related objects. Action detection testing results from video streams demonstrated an F1-score of 0.917, which dropped to 0.863 when interference activities were added. Additionally, the model detected the start of drinking activities with an average latency of 0.24 seconds and the end with 0.04 seconds, indicating high temporal accuracy. These results demonstrate the feasibility of egocentric, vision-based fluid-intake detection and its potential application in preventing dehydration. To our knowledge, this is the first vision-based dataset focusing on fluid-intake actions from a first-person viewpoint-offering a novel foundation for advancing hydration monitoring in older adults and various real-world contexts.