SOURCE-LINKED INTELLIGENCE

DARTS: Decoder-Aware Representation Tuning via Surgery for Model Merging

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Model merging combines multiple task-specific fine-tuned LLMs into a single multi-task model without additional training. However, merged models are known to suffer from representation bias: systematic drift between the merged model's hidden states and those of each individual source model. Prior work (Yang et al., 2024a) study and mitigate this bias for encoder-based vision models using a lightweight correction module trained with L1 loss. However, such bias is not studied for decoder models due to their autoregressive nature. We analyze the problem of representation bias in decoder models, a

Read original source ↗ Open in workspace

recordType

paper

region

Global