SOURCE-LINKED INTELLIGENCE

What Do Tabular Foundation Models Compute In Context? In-Situ Representation Refinement through Attention-Gated Updates

arXiv · AI, language, vision and robotics · article · Sep 23, 2026 · UTC

What reusable computation should a tabular foundation model learn when every table defines a new supervised task? We develop in-situ representation refinement: support labels guide updates to the episode's representations, and these updates transfer to unlabeled queries without changing model parameters. A regularized leave-one-out objective yields a support correction and its query extension. The leading term separates attention-based reading from state-dependent scaling, motivating RefineICL: an attention-gated, FFN-free contextual stack with selected low-rank feature interaction and typed m

Read original source ↗ Open in workspace

recordType

paper

region

Global