Improving large models with small models: Lower costs and better performance.

Abstract

Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have discouraged most product teams from running or fine-tuning them. In such cases, to harness the exceptional performance of PLMs, one must rely on paid Application Programming Interface (API), thereby exacerbating the economic burden. Despite the overall inferior performance of small models, in specific distributions, they can achieve comparable or even superior results. Consequently, some input can be processed exclusively by small models. On the other hand, certain tasks can be broken down into multiple subtasks, some of which can be completed without powerful capabilities. Under these circumstances, small models can handle the simple subtasks, allowing large models to focus on challenging subtasks, thus improving the performance. Specifically, we propose Data Shunt⁺ (DS⁺), a general paradigm for collaboration of small and large models. DS⁺ first trains small models for different tasks and routes user queries based on the confidence levels of small models, allowing both the small and large models to handle the queries they are best suited for. DS⁺ not only substantially reduces the cost associated with querying large models but also effectively improves large models' performance. For instance, ChatGPT achieves an accuracy of 94.43 % on Amazon Product sentiment analysis, and DS⁺ achieves an accuracy of 95.64 %, while the cost has been reduced to 31.18 %. Besides, experiments also prove that the proposed collaborative-based paradigm can better inject specific task knowledge into PLMs compared to fine-tuning.