A simulated SNP experiment indicates a high risk of over-fitting and false positive results when a predictive multiple SNP model is established and tested within the same dataset.

Abstract

Several relatively small studies have established predictive models for normal tissue radiosensitivity based on multiple SNPs. Even though these models yielded statistically significant results, the models were often inconsistent with each other. This can presumably be attributed to certain methodological problems related to the way these models were established and tested. In order to explore this potential problem, we conducted 10 simulated SNP experiments based on randomly assigned 'SNP genotypes' applied to a set of real clinical data. In 8 out of 10 times, a significant result was found for the model. This clearly demonstrates that the process of fitting the model to the dataset is indeed per se capable of producing nominally significant results. Thus, great caution should be taken when a multiple SNP model is established and tested within the same patient cohort.