A novel language model for predicting serious adverse event results in clinical trials from their prospective registrations.

Abstract

With accurate estimates of expected safety results, clinical trials could be better designed and monitored. We evaluated methods for predicting serious adverse event (SAE) results in clinical trials using information only from their registrations prior to the trial. We analyzed 22,107 clinical trials from ClinicalTrials.gov alongside their summary results. We developed a classifier predicting significant differences in the proportion of participants with SAEs (area under the receiver operating characteristic curve; AUC) across experimental and control arms, and a regression model predicting the proportion of participants with SAEs in the control arms (root mean squared error; RMSE). A transfer learning approach using pretrained language models (e.g., ClinicalT5, BioBERT) was used to build a prediction model. To maintain semantic representation in long trial texts, a sliding window method was applied. The best performing model (BioBERT + Transformer + MLP) had 85.1% AUC when predicting which trial arm had a higher proportion of SAEs. When predicting SAE proportion in the control arm, the same model achieved RMSE of 18.8%. The sliding window approach consistently outperformed direct comparisons; the average absolute AUC increase was 3.5%, and absolute RMSE reduction was 1.58%. Classifiers built using language model-based representations consistently outperformed baseline models trained solely on structured data, with an average AUC difference of 11.60%. Summary results data from ClinicalTrials.gov remains underutilized. Predicted results of publicly reported trials provides an opportunity to identify discrepancies between expected and reported safety results.