An enhancing framework with an emphasis on decision balance in ensemble regression.

Abstract

Current ensemble regression learning (ERL) faces two primary limitations: first, traditional fusion strategies overlook the intrinsic complexity of the ensemble's group decision-making (GDM) process; second, the pursuit of diversity exacerbates ensemble instability. These dual constraints have collectively caused ERL research to stagnate at the applied level, impeding further theoretical breakthroughs. In response, this paper proposes a novel ERL framework with decision balance (DBERL), designed to overcome these limitations through a research paradigm that integrates GDM with decision-balanced structures. Specifically, DBERL models the GDM process in traditional ERL as a decision-balanced network (DBN), clarifying both individual-level and group-level decision-making paradigms. Within this network, individuals are adaptively clustered based on task characteristics, thereby forming both narrowly and broadly balanced structures. A hierarchical balanced attention mechanism (HBA) is introduced to aggregate the decision influences of individuals within these structures. Finally, a phased feedback mechanism is incorporated to further promote consensus within the ensemble. The performance of DBERL, including the effectiveness of its internal modules, was rigorously validated across nine diverse datasets, encompassing various application domains, data volumes, and feature dimensions. The results indicate that among 45 evaluation metrics across all datasets, DBERL ranked first in 80% of comparisons against 11 baseline models, in 82.2% of comparisons against 12 ensemble strategies, and in 64% of comparisons against two other balance structures. Based on evaluation results across six dimensions, including fitting capability, correlation, interpretability, stability, data sensitivity, and generalization ability, DBERL achieved the top rank in statistical testing.