Designing reproducible large-language-model-assisted scientific analyses.
Where this comes from
- Record sourced from PubMed, PMID 42746527.
- Also identified by DOI 10.1016/j.patter.2026.101644 and PMC identifier 13576654.
- Licence recorded as CC BY.
- The licence permits redistribution, so the abstract is shown in full and the full text is available from the publisher.
Abstract
Large language models (LLMs) increasingly write code, analyze data, and orchestrate scientific workflows. This can create reproducibility challenges because LLMs blur the boundary between how an analysis is built and what it depends on at run time. Guidelines for LLM-assisted science begin with a critical choice: whether the LLM sits <i>on</i> the data path of the published analysis, a live step results depend on, or <i>off</i> the data path, producing durable artifacts such as code. Reproducible analyses require preserving data, code, and runtime; an on-path LLM becomes part of the runtime, a dependency that may change, be deprecated, or become inaccessible. We derive six recommendations: (1) keep the LLM off the data path where possible, (2) preserve LLM-generated artifacts, (3) verify results by methods suited to the LLM's role, (4) consider open-weight models, (5) record the model version, and (6) assess determinism.