AIIC AI Intelligence Centre

SOURCE-LINKED INTELLIGENCE

How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text

arXiv · AI, language, vision and robotics · article · Sep 30, 2026 · UTC

Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31.1% by August. Unlike synthetic data or model-collapse setups, this *wild* AI text comes from many models, is written for human readers, and arrives unlabeled in pretraining corpora. How does AI text in the wild affect language model pretraining? To answer this question, we pretrain 800 language models, varying the ratio of added AI tokens to human tokens, and fit s

Read original source ↗ Open in workspace

recordType
paper
region
Global

Evidence & attribution

First collected: 2026-10-01T15:32:10.694Z. This is not the publication date.