Assessing the Credibility of Quantitative Information: A General Framework.

Abstract

Quantitative information can be obtained through measurement, inference, or prediction. Each of these approaches has its own theoretical framework for assessing the credibility of the information produced: metrology for measured information, statistics for inferred information, and Computational Science and Engineering for predicted information, particularly through the Verification, Validation, and Uncertainty Quantification framework. These credibility assessment frameworks are mature and well-established, and they are especially important in fields such as in silico medicine, where reliable information is essential to support clinical decision-making. However, intensive research in this area is generating new classes of information estimators that do not fit neatly into any of these three categories. Examples include in silico-augmented clinical trials, physics-informed machine learning predictors, and the use of synthetic datasets to train machine learning models. In this letter, we propose a generalisation of these three distinct approaches to the problem, called the 7S Framework, which addresses this gap by integrating and generalising the credibility assessment approaches used in metrology, statistics, and Computational Science and Engineering. We applied the 7S Framework to seven diverse use cases; the results show that the framework is effective, sufficiently general, and capable of capturing the subtle differences that shape the concept of credibility across different types of quantitative information.