The AI Code Feedback Loop: How Today's Generated Code Trains Tomorrow's Models

About 41% of all new code merged globally in 2025 was AI-generated. Most of that code flows into production repositories that are publicly indexed, scraped, and eventually fed back into the next round of training data for AI coding tools. The implication is straightforward but its consequences are still unfolding: AI models are increasingly being trained on the outputs of prior AI models, with no structured record of which code came from where.

This is the context pollution problem. It is not hypothetical. The feedback loop is already operating at scale, the quality effects are measurable, and the failure mode is unusual enough that it can look like improvement in the short term while the underlying distribution quietly degrades.

The Feedback Loop, Concretely

When a developer accepts a Copilot suggestion, that code typically goes into a git repository. If the repository is public, it is accessible to future training pipelines. Even private repositories get captured indirectly through package registries, open-source mirrors, and the growing share of Stack Overflow answers, documentation, and blog posts that themselves quote or repurpose AI-generated code.

The 2024 Nature paper by Shumailov et al. is the foundational result here. The researchers trained models recursively on their own outputs and observed what they called "model collapse": a progressive loss of the tail distributions in the training data. Rare APIs, edge-case patterns, domain-specific idioms — anything that appeared infrequently in the original corpus — became underrepresented faster than the core mass of common patterns. The effect was not architecture-specific; it showed up in LLMs, variational autoencoders, and Gaussian mixture models.

The mathematics are straightforward. For each recursive training iteration, variance in the learned distribution grows as Var(X^n) = σ²(1 + n/M), where M is the dataset size. The distribution doesn't collapse all at once — it narrows. Each generation produces slightly blander outputs, slightly less capable on minority-class tasks, slightly more prone to confidently wrong outputs at the edges. You might not see it on HumanEval. You will see it when the model is asked to implement something unusual.

What makes this particularly dangerous is the early stage of collapse. Benchmarks on common tasks can improve while the model loses competence on minority data. The overall benchmark score hides the degradation that matters most.

What "Polluted" Code Actually Looks Like

The quality degradation from AI-generated code isn't random. Each major LLM leaves measurable stylistic fingerprints that researchers can now attribute with over 92% accuracy in controlled settings. These fingerprints are not just cosmetic — they represent systematic biases that compound when the model trains on its own outputs.

The patterns researchers have documented include:

• Deprecated API usage: LLMs use deprecated APIs at rates of 25–38% on general code tasks, spiking to 70–90% when the prompt context involves legacy code. This is not a knowledge gap that newer models fix — larger models actually have higher deprecated usage rates, because they memorize more patterns from historical repositories. When that code enters production and training, the model's preference for deprecated patterns gets reinforced.

• Code clones: GitClear analyzed 211 million changed lines of code from repositories at large tech companies between 2020 and 2024. Copy/paste code increased 4x. For the first time in the dataset's history, "copy/paste" exceeded "moved" code — a structural reversal of the refactoring norm that good software engineering had established over decades.

• Refactoring collapse: Refactoring activity dropped from 25% of changed code in 2021 to under 10% in 2024, exactly coinciding with widespread AI coding tool adoption. AI-generated code tends to duplicate rather than abstract, because pattern-matching to the most common local solution is cheaper than awareness of the full codebase's structure.

• Increased churn: Code revised within two weeks of initial commit grew from 3.1% (2020) to 7.9% (2024). The code ships but doesn't stabilize.

When this code re-enters training, the next model learns that copy/paste duplication is normal, that deprecated APIs are still in common use, and that minimal refactoring is the expected style. The model becomes a better predictor of the median 2024 repository — which is already a degraded signal from the median 2020 repository.

Why Security Gets Worse Iteratively

The feedback loop has a particularly measurable effect on security. A controlled experiment ran 400 code samples through 40 rounds of iterative AI-on-AI improvement without human intervention. After just five rounds:

• Critical vulnerabilities increased by 37.6%
• The iteration pattern went from 2.1 vulnerabilities/sample early to 6.2 vulnerabilities/sample by round 8–10
• Security-focused prompts introduced new vulnerabilities even when explicitly requesting security improvements

The mechanism is that AI models hallucinate function names, package names, and security APIs with plausible-sounding but incorrect behavior. Approximately 20% of sampled AI-generated code references hallucinated package names — "slopsquatting" packages that attackers have begun registering to intercept installations. Each AI-generated call to a nonexistent security helper is a confident demonstration that such helpers exist, which trains the next model to suggest them.

A Fortune 50 enterprise tracked this in production: AI-assisted development increased velocity 4x while simultaneously increasing monthly security findings from roughly 1,000 to over 10,000. The velocity is real. So is the cost.

The Detection Problem

The natural response is to add detection: identify AI-generated code and either gate it out of training data or require additional human review. The problem is that detection at the quality level is much harder than it looks.

AI-generated code passes standard linters and syntax checkers. The anti-patterns — over-commenting, single-use helper functions, code duplication, deprecated APIs — all appear in human-written beginner code too. There is no syntactic marker that distinguishes "AI wrote this badly" from "a junior engineer wrote this badly."

Behavioral fingerprinting of AI coding agents (as distinct from the code itself) is more tractable. Researchers can identify Claude Code, GitHub Copilot, and other agents with high accuracy by examining commit metadata, PR descriptions, and comment patterns. But this detects which agent was used, not whether the resulting code is high quality, and fingerprinting accuracy degrades as model families converge toward similar training corpora.

Cryptographic watermarking — embedding imperceptible signals in AI output — faces a different limitation: it is trivially defeated by common code transformations. Renaming a variable, reformatting a block, or refactoring a function breaks most watermarking schemes. The EU AI Act now requires machine-readable provenance marks for general-purpose AI models, but the technical infrastructure to enforce this at the code level does not yet exist.

What Actually Preserves Training Signal

The theoretical result from 2026 (arXiv:2602.16065) is encouraging: model collapse can be avoided if real human-generated data is accumulated alongside synthetic data rather than replaced by it. The convergence rate equals the minimum of the baseline convergence rate and the fraction of real data per iteration. In practice, this means the intervention is at the data curation layer, not the generation layer.

Training data quality has measurable leverage. One study found that cleaning low-quality code from training data reduced low-quality code generation rates from 5.85% to 2.16% — an odds ratio of 6.37 — with no functional correctness penalty. You don't have to choose between quality and capability; quality training data improves both.

For teams building production AI coding systems, a few concrete practices emerge from the research:

• Provenance metadata as a first-class concern. Record which model generated a code block, at what version, and whether a human reviewed the output before merge. Enforce this through CI gates — not developer awareness — because it's the only approach that scales. The VS Code co-author controversy in April 2026 (where a default was silently changed to auto-attach Copilot authorship to all commits) shows how contested this space is, but the direction is clear: provenance metadata will be required.

• Human review gates for AI-generated code, particularly on security-sensitive paths. The 2.74x higher security vulnerability rate in AI co-authored PRs is consistent across multiple independent studies. Automated linting helps but does not substitute for a human examining whether the logic is actually correct.

• Training corpus curation based on post-review state. The distinction between "AI suggested this" and "a human reviewed and accepted this" is the most important training signal available for fine-tuning. Accepted post-review commits are the closest thing to labeled high-quality data in a production codebase.

• Anti-pattern monitoring at the corpus level. Code clone rates, refactoring ratios, and churn rates can track whether AI adoption is systematically degrading codebase quality. These metrics don't require identifying individual AI-generated lines — they reveal structural effects in aggregate.

The Structural Constraint

Epoch AI projects that high-quality, human-generated public text will be exhausted between 2026 and 2032. Over 28% of major corpus sources now block AI crawlers. The pressure toward synthetic data for training is not going away — it is structural.

This means the codebase hygiene practices described above are not just about the quality of code that ships today. They are about preserving the training signal that future models will need to be useful at all. The highest-quality open-source repositories — the ones that have historically been selected as training data precisely because they represent expert engineering judgment — are the ones most likely to be degraded by AI-generated contributions if those contributions are not filtered or labeled.

The feedback loop is already running. The question is whether the data pipeline for the next generation of models will be curated well enough to break the cycle, or whether it will inherit the anti-patterns of the current generation at scale.

The answer depends almost entirely on choices made during code review — not in research labs.