Synthetic data gets pitched as a shortcut around the hard parts of dataset creation. Sometimes it is. Often it is a different set of trade-offs, not a free pass.
Where synthetic data wins
Synthetic data is strong when real examples are rare, sensitive, or expensive to collect, edge cases, low-frequency events, or scenarios involving private information. It also scales fast once a generation process is validated.
Where real data wins
Real data captures the noise, irregularity, and context that synthetic generation tends to smooth over. Models trained only on synthetic data can perform well on synthetic test sets and fail on the messiness of real-world input.
The risk of synthetic-only pipelines
If your synthetic data was generated by a process with its own blind spots, those blind spots get baked into every record. Without a real-data benchmark to check against, you cannot easily tell that this happened.
The practical answer: blend, then validate
Most production datasets that use synthetic data pair it with a real-data validation set. Synthetic data fills coverage gaps. Real data confirms the model still works outside the synthetic distribution.
Treat synthetic data as a tool for filling specific gaps, not a replacement for the discipline of collecting and cleaning real examples. The teams that get burned are the ones who treat it as the latter.