SOURCE-LINKED INTELLIGENCE

When Does Self-Supervised Pretraining Help Tabular Models? A Study of Label Scarcity and Missing Data

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Self-supervised learning (SSL) has emerged as a promising approach for tabular data, yet its efficacy under extreme label scarcity and test-time missingness remains under-explored. In this paper, we evaluate a mask-and-recover SSL pretraining objective against training from scratch and classical baselines across 14 diverse classification tasks. First, while SSL outperforms training from scratch on average and remains competitive with state-of-the-art tree ensembles (achieving ~0.8954 AUC vs. Random Forest's 0.9015 at 10% labels), the SSL-vs-scratch gains exhibit high inter-task variance and la

Read original source ↗ Open in workspace

recordType

paper

region

Global