MultiTabPFN: Codebook-based extensions of TabPFN for high-class-count tabular classification.

Abstract

Tabular data are among the most common data types, and TabPFN has recently emerged as a powerful foundation model offering fast, training-free predictions. However, its applicability to high-class-count classification remains limited, as fine-tuning or retraining incurs heavy computational costs. We address this gap by framing multiclass prediction within the Error-Correcting Output Codes (ECOC) paradigm, a training-free approach whose effectiveness depends critically on codebook design and decoding. We present the first systematic study of ECOC-based extensions for TabPFN and introduce MultiTabPFN, a modular framework with Classwise Principal components-based Indexing (CPI)-a novel codebook method that encodes class-level geometry into compact binary codes. Compared to the conventional ECOC constructions, CPI explicitly balances separability and redundancy in the code space, thereby providing a principled path for scaling tabular foundation models to many-class settings. Combined with confidence-aware decoding, MultiTabPFN consistently outperforms standard ECOC baselines across synthetic tasks and 36 real-world benchmarks, establishing a practical and training-free extension of TabPFN to high-class-count tabular classification.