PEER: Towards reliable and efficient inference via Patience-Based Early Exiting with Rejection.

Abstract

In biomedical applications, models must balance inference efficiency with reliable predictions. Patience-based early exiting (PABEE) accelerates inference but often fails under uncertainty. We propose PEER (Patience-based Early Exiting with Rejection), a unified framework that integrates a rejection mechanism into PABEE to enable both efficiency and reliability. With PEER, models make decisions for the input or reject by employing a patience counter to track prediction consistency across layers. This design avoids unreliable final-layer predictions and unifies early exiting with selective abstention without retraining. We evaluated PEER on 11 biomedical datasets, including clinical text and medical images. Experiments involved multiple Transformer-based backbones, including vision transformer, measured by accuracy, macro-F1, and speed-up ratio. Experiments demonstrate that PEER consistently improves reliability while preserving the efficiency gains of early exiting. For instance, on the MIMIC-III dataset, PEER achieves an accuracy of 90.73% (surpassing the baseline of 89.48%) by rejecting only 2.79% of uncertain samples. Alternatively, in high-efficiency settings, it achieves 80% speed-up ratio while maintaining comparable performance. Across diverse datasets, PEER successfully abstains from uncertain cases that baseline methods misclassify, leading to more trustworthy predictions. It generalizes effectively across different model architectures, scales, and modalities, showing robustness in both language and vision tasks. Case studies further confirm that PEER aligns with clinical workflows by deferring ambiguous cases for human review. PEER offers a simple, architecture-agnostic framework that jointly ensures fast and trustworthy inference. Its generalizability across language and vision models highlights strong potential for deployment in clinical decision support.