Towards a Multi-Stakeholder process for developing responsible AI governance in consumer health.

Abstract

AI is big and moving fast into healthcare, creating opportunities and risks. However, current approaches to governance focus on high-level principles rather than tailored recommendations for specific domains like consumer health. This gap risks unintended consequences from generic guidelines misapplied across contexts and from providing answers before agreeing on the questions. Our objective is to explore pragmatic multi-stakeholder approaches to govern consumer-facing health AI. The aims are to (1) establish an approach tailored for consumer health AI governance and (2) identify key constraints and desirable model characteristics. This paper synthesizes insights informed by a 4-month multidisciplinary expert consensus process with nearly 200 participants. The deliberations provided guidance for the development of the proposed governance models in consumer health AI. (1) A Shared View of Consensus: A process for consumer health AI governance should limit the scope and incorporate multi-stakeholder perspectives centered on patient needs. Desirable model characteristics include adaptability, patient empowerment, and transparency. (2) Recommended Collaborative Process: A pathway for effective governance should begin by forming a Health AI Consumer Consortium (HAIC₂) representing patients and aligning incentives across stakeholders. While examples focus on the United States healthcare system, core themes around incorporating consumer voices, enabling transparency, and balancing innovation with thoughtful oversight while avoiding overambitious scope will have relevance globally. As consumer AI spreads worldwide, the multi-stakeholder alignment and patient empowerment principles proposed here may offer productive ways to ensure AI for consumers is safe, effective, equitable, and trustworthy (SEET).