Large Language Models Are Multitask Chain-of-Thought Prompting Optimizers.

Abstract

Large language models (LLMs) have achieved striking performance across a broad range of reasoning benchmarks, yet the quality of their outputs remains acutely sensitive to prompt design. A meticulously engineered prompt can coax an LLM into correctly answering even highly complex questions, prompting a surge of research into techniques that boost prompt efficacy. Manual chain-of-thought (CoT) prompting and automated prompt generation have emerged as leading strategies. However, CoT exemplars must be painstakingly tailored to each task, making the process labor-intensive, while prompts optimized for a single task often fail to generalize. We introduce a simple yet powerful alternative: by treating the LLM itself as a multitask optimizer, we enable iterative self-refinement of prompts through natural language task descriptions and few-shot in-context learning (ICL). Recognizing that individual prompts exhibit task-dependent sensitivity, we further ensemble the top-performing prompts at inference time. Empirical evaluation with several state-of-the-art LLMs shows that our method substantially surpasses prior baselines, delivering gains of up to 6.0% on mathematical reasoning tasks and 10.2% on commonsense reasoning benchmarks.