Assessing the potential utility of large language models for assisting community health workers: protocol for a prospective, observational study in Rwanda.

Abstract

Community health workers (CHWs) are critical to healthcare delivery in low-resource settings but often lack formal clinical training, limiting their decision-making. Large language models (LLMs) could provide real-time, context-specific support to improve referrals and management plans. This study aims to evaluate the potential utility of LLMs in assisting CHW decision-making in Rwanda. This is a prospective, observational study conducted in Nyabihu and Musanze districts, Rwanda. Audio recordings of CHW-patient consultations will be transcribed and analysed by an LLM to generate referral decisions, differential diagnoses and management plans. These outputs, alongside CHW decisions, will be evaluated against a clinical expert panel's consensus. The primary outcome is the appropriateness of referral decisions. Secondary outcomes include diagnostic accuracy, management plan quality, and patient and user perceptions to ambient recording of consultations. Sample size is set at 800 consultations (400 per district), powered to detect a 15-20 percentage point improvement in referral appropriateness. Ethical approval has been obtained from the Rwandan National Ethics Committee (RNEC) (Ref number: RNEC 853/2025) in June 2025, recruitment started in July 2025 and results are expected in late 2025. Results will be disseminated via stakeholder meetings, academic conferences and peer-reviewed publication. PACTR202504601308784.