What Provider Frequently Asked Questions Miss: Evaluating Unmet Attention-Deficit/Hyperactivity Disorder Information Needs Through Comparison of Online Community Posts Using Large Language Model-Assisted Semantic Analysis in a Mixed Methods Study.

Abstract

Attention-deficit/hyperactivity disorder (ADHD) is a prevalent neurodevelopmental disorder that affects the functioning and quality of life of individuals throughout their lifespan. Despite the extensive information available online, patients and caregivers continue to report unmet needs, particularly regarding diagnosis, treatment, medication effects, comorbidities, and long-term management strategies. Existing provider-generated frequently asked questions (FAQs) are widely used, but often fail to fully capture the concerns expressed in online communities. This study aimed to (1) evaluate the extent to which provider-generated ADHD FAQs cover questions from online communities, (2) identify unmet information needs by analyzing questions with low semantic similarity to FAQs, and (3) compare the response styles of provider-generated answers with community-generated answers through large language model (LLM)-assisted analysis. ADHD-related questions from a Korean online community were semantically compared with provider-generated FAQs using sentence embedding-based similarity analysis to assess coverage and identify matched versus unmatched questions. Unmatched questions underwent topic modeling using the LimTopic framework, integrating BERTopic with LLM-assisted summarization to uncover unmet needs. An LLM-assisted content analysis was conducted on the answers to the high-similarity FAQ-community question pairs, enabling an examination of the response styles used by each group when addressing the public. Through similarity comparison using embedding models and manual verification, the paraphrase-multilingual-MiniLM-L12-v2 (MBERT) model, which achieved the highest <i>F</i>₁-score of 0.45, was selected as the final embedding model. The optimal similarity threshold determined for this model was 0.766, and the coverage of questions with similarity above this threshold between FAQs and the online community was 52.09% (2598/4988). Most of the coverage was concentrated on 18 FAQs. Online community questions below the similarity threshold were reviewed by experts after LimTopic analysis, resulting in the identification of 12 categories of unmet consumer needs, including school and social support, treatment accessibility, psychological support, and comorbidity management. Response style analysis revealed significant differences between evidence and authority signaling and the actionability dimension. Provider-generated ADHD FAQs covered approximately half of consumers' questions, revealing substantial gaps in information provision for patients with ADHD. Health information on ADHD should expand beyond basic medical knowledge to address consumers' real-world experiences, including access to care, school and social support, evidence-based treatments, daily functioning strategies, psychological support, health care navigation, and comorbidity management. Integrating these elements into provider-generated FAQs can create more comprehensive, consumer-centered resources that better support informed decision-making and long-term self-management.