Development and feasibility testing of an artificially intelligent chatbot to answer immunization-related queries of caregivers in Pakistan: A mixed-methods study.

Abstract

Gaps in information access impede immunization uptake, especially in low-resource settings where cutting-edge and innovative digital interventions are limited given the digital inequity. Our objective was to develop an Artificially Intelligent (AI) chatbot to respond to caregiver's immunization-related queries in Pakistan and investigate its feasibility and acceptability in a low-resource, low-literacy setting. We developed Bablibot (Babybot), a local language immunization chatbot, using Natural Language Processing (NLP) and Machine Learning (ML) technologies with Human in the Loop feature. We evaluated the bot through a sequential mixed-methods study. We enrolled caregivers visiting the 12 selected immunization centers for routine childhood vaccines. Additional caregivers were reached through targeted text message communication. We assessed Bablibot's feasibility and acceptability by tracking user engagement and technological metrics, and through thematic analysis of in-depth interviews with 20 caregivers. Between March 9, 2020, and April 15, 2021, 2,202 caregivers were enrolled in the study, of which, 677 (30.7%) interacted with Bablibot (users). Bablibot responded to 1,877 messages through 874 conversations. Conversation topics included vaccination due dates (32.4%; 283/874), side-effect management (15.7%;137/874), or delaying vaccination due to child's illness or COVID-lockdown (16.8%;147/874). Over 90% (277/307) of responses to text-based exit surveys indicated satisfaction with Bablibot. Qualitative analysis showed caregivers appreciated Bablibot's usefulness and provided feedback for further improvement of the system. Our results demonstrate the feasibility and acceptability of local-language NLP chatbots in providing real-time immunization information in low-resource settings. Text-based chatbots canminimize the workload on helpline operators, in addition to instantaneously resolving caregiver queries that otherwise lead to delay or default.