Improving Conversational Literature Retrieval Quality via Personalized Profile-Based Re-ranking.

Abstract

Academic literature retrieval is constrained by the paradox of "information overload" versus "evidence scarcity", a tension that deepens when researchers iteratively refine their queries in multi-turn conversational settings. To address this challenge, we propose Conversational Literature Personalized Re-ranking (CLPR), a personalized framework that unifies dense semantic retrieval with personalized user profiling. CLPR first performs a broad high-recall retrieval to collect candidate documents, then compresses conversational history into a concise textual profile that encodes sequential continuity, immediate focus, and long-term research background via a large language model. The generated profile serves as a pseudo-query for a neural cross-encoder to produce the final ranking. Cross-domain testing on the public LitSearch (computer science) benchmark confirms its robust generalization, yielding an NDCG@10 of 0.4793. On MedCorpus, a new multi-turn biomedical conversational retrieval benchmark constructed for this study, CLPR attains state-of-the-art performance with P@1 = 0.9497 and NDCG@10 = 0.9271, surpassing the strongest baseline by substantial margins. Ablation shows long-term background cues contribute most, and maintaining a short, up-to-date profile across turns outperforms a static one. CLPR therefore delivers accurate, personalized literature retrieval and can accelerate evidence synthesis across scientific domains.