SOURCE-LINKED INTELLIGENCE

Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning

arXiv · AI, language, vision and robotics · article · Sep 22, 2026 · UTC

Music creation often involves iterative refinement, changing selected musical details while retaining the rest. To support such refinement, we introduce SpanSynth-Edit, a flow-matching model for MIDI-guided synthesis and editing of multi-instrument audio mixtures using low-frame-rate scalar-quantised latents. MIDI Span encodes instrument-labelled note lifecycles as unordered event sets with continuous-valued attributes and pools each set into one conditioning vector per audio-latent frame. The model uses contextual audio for instrument-specific timbre guidance and supports editing by resynthes

Read original source ↗ Open in workspace

recordType

paper

region

Global