ARTIFICIAL INTELLIGENCE<\/p>\n
The internet’s high-quality data is effectively exhausted for training frontier AI models. That’s the consensus. The proposed solution is synthetic data \u2014 and it deserves more scrutiny than it’s getting.<\/strong><\/p>\n The idea is elegant: if you’ve run out of real data to train on, use AI to generate synthetic data that captures the same statistical properties. Train the next generation of models on a combination of real and synthetic data. Problem solved.<\/p>\n Except it’s not quite that simple.<\/p>\n Synthetic data genuinely works well for specific, structured tasks where the distribution of correct answers is well-defined. Mathematical proofs, coding problems, logic puzzles, scientific calculations \u2014 these can be synthetically generated at scale and used to meaningfully improve model performance on those tasks. The reasoning model improvements of 2025 were largely built on synthetic data of this type.<\/p>\n “Training a model on its own outputs is like photocopying a photocopy \u2014 the errors compound while the signal degrades.”<\/p><\/blockquote>\n The deeper issue is model collapse \u2014 what happens when you train models on synthetic data generated by earlier models. Training a model on its own outputs is like photocopying a photocopy. The errors compound, the edge cases get smoothed away, the diversity of the distribution degrades, and you end up with a model that is less capable than what you started with in ways that are subtle and hard to detect.<\/p>\n Research on this is still active, and the magnitude of the problem is contested. But the risk is real, and the field’s confidence that synthetic data simply solves the training data problem deserves more scepticism than it’s currently receiving.<\/p>\n Tags: Artificial Intelligence \u2022 Opinion \u2022 Technology & Society \u2022 192.168.1.22\/<\/p>\n","protected":false},"excerpt":{"rendered":" Synthetic data is being positioned as the solution to the data wall problem. It’s a genuine tool \u2014 but the limitations are bigger than the hype suggests.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,23],"tags":[],"class_list":["post-9018","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence","category-technology"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/posts\/9018","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/iamglover.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=9018"}],"version-history":[{"count":1,"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/posts\/9018\/revisions"}],"predecessor-version":[{"id":9090,"href":"http:\/\/iamglover.com\/index.php?rest_route=\/wp\/v2\/posts\/9018\/revisions\/9090"}],"wp:attachment":[{"href":"http:\/\/iamglover.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=9018"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/iamglover.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=9018"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/iamglover.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=9018"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}What Synthetic Data Is Good At<\/h2>\n
The Model Collapse Problem<\/h2>\n