Developing and evaluating human-led and large language model-assisted hybrid deductive-inductive workflows for qualitative analysis.

Abstract

To develop and systematically compare a human-led and LLM-assisted hybrid deductive-inductive workflow for qualitative analyses. We analyzed 122 transcripts (n = 61 research clinical consultations; n = 61 reflexive interviews) from a video ethnography study of patients with heart failure. Human-led thematic analysis used Dedoose software, and LLM-based analysis was conducted using ChatGPT Edu (GPT-5.2; OpenAI) with an eleven-prompt protocol. Both applied a hybrid deductive-inductive approach. The research team compared outputs across 63 human-LLM theme pairs using human consensus and LLM-based evaluation, integrated themes into a final framework, and manually verified quotation fidelity against the original transcripts. Human-led and LLM-generated analyses produced complementary cross-cutting themes (7 human; 9 LLM), all judged valid and integrated into 14 final themes across three domains. Thematic overlap was moderate to substantial (Hit Rate 1.00; Jaccard 0.44-0.51). Robustness testing across three runs revealed recurrence of five core concepts alongside variability in theme labels and counts. Quotation fidelity showed 68% verbatim, 20% paraphrased, 6% partial and 3% full hallucinations, and 3% truncated excerpts; verbatim quotations did not always clearly support their assigned themes. LLM-assisted analysis is feasible for large-scale qualitative health research within a HIPAA-compliant environment using an adaptable eleven-prompt protocol. Human oversight remained essential for contextual interpretation, quotation verification, and assessment of theme-quotation support. LLMs are best positioned as analytic partners rather than autonomous coders. Transparent workflows with human-in-the-loop validation are essential for responsible AI integration in health and biomedical informatics.