Every large language model today begins its life by ingesting hundreds of billions of words scraped from the public web. But what happens when the web itself is quietly weaponized? A new paper from researchers at the University of Washington and the Allen Institute for AI demonstrates that poisoning LLM pretraining data at web scale is not just theoretically possible, it is already practical through a mechanism that every AI founder should understand: public discussion interfaces.

The paper, authored by Victoria Graf, Hannaneh Hajishirzi, Noah A. Smith, David Kohlbrenner, and Kyle Lo, introduces a concept called HalfLife that changes how we think about data safety in the age of web-scale AI training.

The Poison Vector Nobody Is Watching

Prior research on data poisoning has focused on controlled environments like Wikipedia editing, where a malicious actor could insert subtle errors into a known dataset. But this misses the real threat. Modern pretraining corpora are assembled by crawling billions of web pages through Common Crawl and similar pipelines. The data comes from forums, comment sections, product reviews, blog discussions, and social media. Every publicly writable interface on the web is a potential injection point for adversarial content.

The researchers demonstrate that an attacker does not need to compromise a major website. They simply need to post content in public discussion threads that web crawlers will eventually index. A coordinated campaign across Reddit threads, Hacker News comments, GitHub issue discussions, and Disqus-powered blog comments can inject thousands of subtly poisoned passages into the pretraining pipeline. The scale is what makes this dangerous. A single actor running 100 bot accounts across 10 platforms can generate millions of words of poisoned content in a week.

HalfLife: Measuring Poison Persistence in Training Data

The paper's key technical contribution is HalfLife, a novel metric that estimates how long adversarial content survives in a pretraining corpus after web crawling and data curation. The name is intentionally grim. Just as radioactive isotopes decay over time, poisoned data decays through the deduplication, filtering, and quality scoring stages of a typical data curation pipeline. HalfLife measures the half-life of injected content how much of the poison survives each stage.

The findings are sobering. Common data curation techniques like MinHash deduplication and perplexity filtering catch some adversarial content, but they miss carefully crafted injections that mimic natural language patterns. Content written to match the statistical distribution of legitimate training data slips through. The HalfLife metric reveals that sophisticated attacks can achieve a half-life of 60 to 80 percent meaning 60 to 80 percent of injected content survives the full curation pipeline and ends up in the training set.

For context, if an attacker injects 10 million tokens of poisoned data across web forums, roughly 6 to 8 million tokens will survive deduplication and filtering. Against a 15 trillion token training corpus this sounds small, but the researchers show that even 0.01 percent poisoned data is enough to introduce measurable behavioral changes in a 70 billion parameter model.

Why This Matters More Than Traditional Data Poisoning

Previous data poisoning research focused on targeted attacks against specific models or fine-tuning datasets. A poisoned Wikipedia article about a specific person could bias a QA model. But computational propaganda attacks are different. They operate at the scale of the entire pretraining corpus and they target general model behavior, not specific facts.

The attack surface is enormous. Every public web forum, every comment section, every review platform that a web crawler indexes is a potential injection point. The researchers map the landscape and find over 200 million publicly writable web pages that Common Crawl regularly indexes. Even if only a fraction of these are exploited, the volume of poisonable content is staggering.

What makes this particularly concerning for the current AI landscape is the timing. Companies are racing to build larger and larger models, which means they are consuming more web data than ever. The pressure to scale pretraining datasets creates a natural blind spot for data quality. When your goal is 15 trillion tokens, you optimize for quantity over scrutiny. The HalfLife framework provides a way to measure this risk, but most AI companies are not using it yet.

What This Means for Builders

For founders building AI products, this paper carries three immediate implications. First, if you are fine-tuning a base model that was pretrained on web-scale data, your model may already contain subtle biases or vulnerabilities from computational propaganda attacks. Red-teaming your model against known attack patterns is no longer optional, it is a basic hygiene practice.

Second, the HalfLife metric is directly applicable to your own data pipelines. If you are curating training data from web sources even at a small scale you should measure the half-life of your deduplication and filtering stages. Run a controlled injection test on a sample of your pipeline to understand how much adversarial content survives.

Third, this research opens a new category of product opportunity. Tools that measure data poisoning risk, audit training corpora for adversarial content, and provide HalfLife scores for data pipelines will become essential infrastructure as models grow larger and web data becomes the default pretraining source. The researchers have open-sourced their analysis framework, and the team that productizes it for the enterprise market could own a critical piece of the AI safety stack.