Synthetic Data: The Quality Paradox

Synthetic data lets you train without paying for human labels. It also can quietly degrade quality if you do not handle it carefully. The paradox of generating your own training set.

Why use it

Synthetic data, generated by another model rather than collected from humans, is cheaper and more controllable than real data. It scales to volumes real data can't reach (millions of structured examples in a day). It can target specific weaknesses (give me 10,000 examples of edge case X). For tasks where real data is scarce or expensive (rare languages, regulated domains, niche skills), synthetic is often the only viable scaling path.

The volume advantage. Generating a million examples takes hours; collecting a million from humans takes months. For tasks where you need scale to drive performance (instruction tuning, code generation), synthetic is the only economically feasible path.

The targeting advantage. Real data has whatever distribution it has. Synthetic data lets you over-sample the cases that matter, adversarial, edge-case, rare-skill examples can be generated on demand. The targeting is what makes synthetic data more efficient per example than random real data.

The privacy advantage. Real data has privacy and legal constraints. Synthetic data, generated from prompts, can avoid PII entirely (when carefully designed). For regulated industries, this is the difference between a usable dataset and an unusable one.

The honest caveat. Synthetic data is not a free lunch. The costs come later, in subtler ways than expected. Understanding the costs is what separates teams who use synthetic data well from teams whose models silently degrade.

The collapse risk

Models trained on too much synthetic data lose diversity and shift toward the generator's biases. The technical name is "model collapse", output distributions narrow over generations of synthetic-on-synthetic training. The danger is sneaky: collapse doesn't show up in standard accuracy metrics; it shows up in "the model is fluent but bland; can't generate novel solutions; produces outputs that all sound the same".

The mechanism. The teacher model has a distribution over outputs. Synthetic samples concentrate in the high-probability regions; the long tail is undersampled. Training a student on synthetic data gives the student a narrower distribution than the teacher. Iterate this and the distribution collapses progressively.

The benchmark blindness. Standard benchmarks measure typical-case performance. The collapse happens in the long tail. Models with collapsed distributions can score within 1-2% of pre-collapse on benchmarks while being noticeably less creative or generalising worse on novel inputs.

The detection. Track output diversity metrics (lexical diversity, semantic diversity via embedding clustering, distinct n-gram coverage). Track perplexity on out-of-distribution inputs. If diversity drops or OOD perplexity rises while in-distribution metrics hold, you're collapsing.

The dose-response. Collapse risk grows with the synthetic-to-real ratio. <30% synthetic mixed with real data: minimal collapse. 30-70% synthetic: noticeable but recoverable. >70% synthetic: substantial collapse risk over multiple training generations.

Mitigations

Mix synthetic with substantial real data. Use multiple generators with different biases. Filter aggressively for quality and diversity. Don't train on data generated entirely by yesterday's checkpoint of the model you're training. The cleaner the data pipeline (with explicit diversity controls), the safer the synthetic at scale.

The mixing principle. Synthetic data should typically be no more than 50-70% of training data, with the rest being real. The real data anchors the distribution; the synthetic data scales it. Pure synthetic invites collapse; mostly synthetic with real anchor avoids it.

The multi-generator principle. Use 2-4 different generator models with different training corpora. The diversity of generators limits the per-generator bias from dominating. Single-generator synthetic data inherits all of that generator's blindspots.

The aggressive filter principle. Filter generated examples for quality (LLM-judge), diversity (embedding-based dedup), and adversarial coverage (does this example test something interesting?). Throwing out 50-80% of generated data is normal; the remaining 20-50% is dramatically higher value.

The "different generation than training" principle. Don't train model M on data generated by checkpoint M-1. Use a substantially different model (different family, different scale). The distribution shift between teacher and student dilutes the iterated-collapse pattern.

Where it works now

Code generation, math reasoning, instruction following, domains where the generator can be verifiably correct. Tasks where you can mechanically check synthetic data quality. Less safe: open-ended writing, subjective tasks, anywhere the generator's stylistic biases would propagate.

The verifiability advantage. Code can be tested. Math has answers. Structured outputs can be schema-checked. For these, synthetic data quality can be measured per-example; bad examples filtered automatically. The filtering is what makes synthetic data work in these domains.

The instruction-following case. Generate (instruction, response) pairs. Verify the response satisfies the instruction (using LLM judge or automated checks). Filter; train. The pattern has produced strong instruction-tuned models with relatively little real instruction data.

The translation case. Generate parallel sentences via back-translation (translate target → source → target; verify consistency). The verification is automatic. Synthetic translation data substantially augments real parallel corpora for low-resource languages.

The dangerous domains. Creative writing, subjective opinions, nuanced reasoning. These lack clean verifiers; synthetic data inherits generator biases without filter. Use real data for these; reserve synthetic for verifiable domains.

Common antipatterns

Self-distillation without diversity. Training the model on its own outputs. Collapses fastest; rarely produces real improvement.

Skipping the diversity filter. Generated data is naturally clustered around prompt themes. Without explicit dedup, you train on near-duplicates which is wasted compute.

Trusting LLM-judge for subjective domains. The judge has biases; the generator has the same biases; agreement is high but quality isn't. Use real human evaluation for subjective tasks.

No held-out real eval. Without a real-data eval, you can't detect collapse. Always reserve real data for evaluation, even if you don't use it for training.

What to do this week

Three moves. (1) For any synthetic data pipeline you have, audit the synthetic-to-real ratio. If >70% synthetic, plan to dilute with more real data or accept higher collapse risk. (2) Add diversity metrics (n-gram, embedding-cluster) to your training monitoring. The numbers are cheap to compute and worth their weight. (3) Verify you have a real-data held-out eval that wasn't generated. Without it, collapse will surprise you in production.