Contrastive diffusion model for exploring mathematical expressions from data.

Abstract

Symbolic regression is the task of discovering mathematical expressions that best fit the given data, which is a challenging problem with significant implications in various real-world applications. Existing deep generative models typically tackle this problem through autoregressive generation. However, these models rely on unidirectional dependencies that may be misaligned with the inherently global and bidirectional relationships among components in mathematical expressions, thereby hindering their effective capturing of the implicit global semantic structure during generation. In this paper, we propose an iterative non-autoregressive approach based on diffusion models, refining the generation process of entire mathematical expressions within a latent space to enhance the modeling of global semantic logic. Furthermore, we incorporate contrastive learning during model training to align the features of data points with those of symbolic expressions. This alignment alleviates modal discrepancies between these two data types, facilitating improved utilization of multimodal data in non-autoregressive generation, while simultaneously equipping the model with a more comprehensive semantic understanding. Experimental results demonstrate that our model significantly outperforms multiple mainstream baselines across various benchmark datasets. Notably, it efficiently explores mathematical expression solutions from the data, achieving superior fitting performance while ensuring a high degree of simplicity. These findings highlight the effectiveness of our approach in modeling mathematical expressions from a global perspective and illustrate its capability to efficiently address the symbolic regression problem.