Necessary and sufficient knowledge enhanced collaborative logical reasoning in LLMs.

Abstract

Large language models (LLMs) learn massive knowledge through pre-training, and they have demonstrated strong reasoning capabilities by leveraging the knowledge. However, LLMs often make mistakes due to the failure of using necessary and sufficient knowledge. The inadequate utilization of sufficient knowledge will lead to the incorrect conclusions due to the lack of important evidence. Unnecessary knowledge may mislead the LLMs into generating false reasoning paths. To tackle the above challenges, we propose a collaborative logical reasoning framework called CLR. We first utilize deductive reasoning based on evidence retrieval to generate reasoning paths. Next, we use abductive reasoning based on knowledge attribution to identify the necessary conditions. Then we use necessary conditions to verify the correctness of reasoning paths and obtain reliable reasoning paths. Finally, we conduct reliable inductive reasoning to obtain the final reasoning conclusion. Therefore, CLR achieves the collaboration of multiple logical reasoning paradigms. Extensive experiments demonstrate that CLR outperforms a series of baselines on multiple datasets. It also performs well in error identification and self-correction. Our work contributes to remedy the inherent limitations of the logical reasoning paradigms in LLMs and lays the foundation for modeling human cognitive thinking.