DPAC: A tri-dimensional coordinated framework for breaking the efficiency ceiling in parameter-efficient fine-tuning.

Abstract

While parameter-efficient fine-tuning (PEFT) has become essential for adapting large pre-trained models, it is challenging to achieve optimal trade-offs between parameter budgets and task performance. The core difficulty lies in reducing parameters and maximizing the utilization efficiency of each allocated parameter across heterogeneous model components. Recent advances like DoRA improve upon LoRA through magnitude-direction weight decomposition. Nevertheless, there is an unexplored limitation: existing methods optimize parameterization structures in isolation but neglect the coordinated dynamics across resource allocation, inter-component collaboration, and training adaptation that collectively determine parameter efficiency. Given this issue, DPAC, a tri-dimensional framework, is proposed. Specifically, (1) dynamic budget allocation prioritizes layers based on their downstream influence rather than uniform distribution; (2) explicit collaborative constraints enforce synergistic updates between decomposed weight components to enhance magnitude-direction coordination; (3) adaptive optimization scheduling progressively intensifies coordination from exploration to convergence phases. Experiments on vision-language retrieval benchmarks (Flickr30k and MSCOCO) with CLIP and BLIP models reveal that DPAC improves Recall@1 by 1.7-2.2% over DoRA while using identical trainable parameters (less than 1.1% of total model parameters), with amplified gains in constrained scenarios: +2.7% at low rank (r=4) and +3.5% with limited training data (10%). To sum up, principled coordination across optimization dimensions, rather than isolated structural innovations, is crucial to breaking the efficiency ceiling in PEFT.