SOURCE-LINKED INTELLIGENCE
Playing log(N)-Questions over Wikipedia Abstracts: How Per-Round Errors Compound Under Information Asymmetry
arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC
We evaluate six frontier language models on the two-agent $\log_2 N$-Questions game (Potash et al., 2019) to measure self-communication across an information asymmetry. A questioner with access to $N$ candidate Wikipedia lead paragraphs ($N = 4$ to $1024$) must identify a secret target using exactly $\log_2 N$ binary questions answered by an agent from the same provider that sees only the target. Across 408 games, win rate decays cleanly as a geometric power of horizon length, $p^{\log_2 N}$ ($p \approx 0.93$). Per-round failure rates are flat across the horizon, indicating that errors compoun
Read original source ↗ Open in workspace
- recordType
- paper
- region
- Global
Evidence & attribution
First collected: 2026-09-19T20:28:26.698Z. This is not the publication date.
Observed changes
AIIC observation times, not verified publisher revision times. Up to eight recent revisions.
2026-09-23T17:51:24.264Z
- title:
Playing log(N)-Questions over Wikipedia Abstracts: Communication Efficiency Between Paired Frontier Models → Playing log(N)-Questions over Wikipedia Abstracts: How Per-Round Errors Compound Under Information Asymmetry - summary:
We evaluate six frontier language models on the two-agent $\log(N)$-Questions game. A questioner sees $N$ Wikipedia lead paragraphs and must identify a secretly chosen target using exactly $\log_2 N$ yes/no questions. An answerer sees only the target and the question, and replies with one word. Both roles run on the same provider, so the game measures how well a model communicates with itself across an information asymmetry. We run 408 games over document sets of 4 to 1024 paragraphs at a total API cost of \$363. One model finishes well behind the others: Claude Opus 5 wins 28 of 68 games, aga → We evaluate six frontier language models on the two-agent $\log_2 N$-Questions game (Potash et al., 2019) to measure self-communication across an information asymmetry. A questioner with access to $N$ candidate Wikipedia lead paragraphs ($N = 4$ to $1024$) must identify a secret target using exactly $\log_2 N$ binary questions answered by an agent from the same provider that sees only the target. Across 408 games, win rate decays cleanly as a geometric power of horizon length, $p^{\log_2 N}$ ($p \approx 0.93$). Per-round failure rates are flat across the horizon, indicating that errors compoun