Bias-aware training and evaluation of link prediction algorithms in network biology.

Abstract

For biomedical applications, new link prediction algorithms are continuously being developed. These algorithms are typically evaluated computationally, using test sets generated by sampling the edges uniformly at random. However, as we demonstrate, this evaluation approach introduces a bias toward "rich nodes," i.e., those with higher degrees in the network. More concerningly, this bias persists even when different network snapshots are used for evaluation, as recommended in the machine learning community. This creates a cycle in research where newly developed algorithms generate more knowledge on well-studied biological entities while understudied entities are commonly overlooked. To overcome this issue, we propose a weighted validation setting specifically focusing on low-degree nodes and present AWARE strategies to facilitate bias-aware training and evaluation of link prediction algorithms. These strategies can help researchers gain better insights from computational evaluations and promote the development of new algorithms focusing on novel findings and understudied proteins.