SOURCE-LINKED INTELLIGENCE

Hybrid GPU-CPU Retrieval for Personalized Search at Ultra-Large Scale

arXiv · AI, language, vision and robotics · article · Sep 18, 2026 · UTC

Embedding-based retrieval on user-generated content at the trillion-document scale exposes a sharp conflict between two production demands: deep, expressive personalization for queries with rich user intent, and broad coverage of a massive inventory under fixed latency and resource budgets. We characterize this as the personalization-scale paradox: hosting the full serving inventory in GPU memory is too resource intensive, while CPU compute cannot execute the same interaction-heavy model on the latency-critical path. We present a hybrid GPU-CPU co-serving system that resolves the paradox throu

Read original source ↗ Open in workspace

recordType

paper

region

Global