Synthetic dataset generation which is actually good (i.e. adds meaningful information by probing a latent space in truly unique ways + very high temperature sampling + including outside feedback for diversity seeding) will solve the "internet is polluted now with LLM outputs" problem
Barring that, mass swarms of AI agents going into the world and retrieving new information, using weak/active learning to mass label, and improving themselves with increasingly less human intervention is INEVITABLE.
The issue with folks like yourselves and frankly most of mainstream ML research today is a failure of imagination. As it turns out, ML/AI Ph.D's seemed to be too busy lying about their models scores on NeurIPS papers to read sci-fi or other speculative fiction which imagines what technological changes will do to society. This a huge risk, as rich people with a lack of moral compass often view works of speculative fiction as blueprints.
Here's a few which imagined just a tiny bit of a world impacted by the exact sorts of AI systems which are moving far faster than we could have ever imagined:
1. https://en.wikipedia.org/wiki/Permutation_City
2. https://en.wikipedia.org/wiki/The_Jetsons
3. https://en.wikipedia.org/wiki/I_Have_No_Mouth,_and_I_Must_Sc...
4. https://en.wikipedia.org/wiki/Erewhon
5. https://en.wikipedia.org/wiki/The_Diamond_Age
6. https://en.wikipedia.org/wiki/Master_of_the_World_(1934_film...
https://en.wikipedia.org/wiki/Ghost_in_the_Shell_(1995_film)
The future is teleological (the academy unjustly treats that word like a slur), and you should listen a bit more to those who fear the AI and warn you about just how wrong you are about the claim that AI "won't go up on an exponential curve"
You think writing "is INEVITABLE" in caps proves something? You think Science Fiction novels prove something about what will happen in the future?
Is this satire? Do you not understand the meaning of the word FICTION?
https://en.wikipedia.org/wiki/Life_imitating_art
Yes, fiction turning to reality is inevitable. LLMs by definition create "artificial life" out of art. I repeat, reality is teleological.
Doesn't anyone think the future is unpredictable anymore? Does your life experience not include things tanking an unexpected turn, ever?
I can't believe how many commenters here and elsewhere take the future trajectory of AI for granted. Where are the flying cars, the moon bases, the nuclear powered appliances? Things don't always go up on an exponential curve, and even when they do, results are not always what we thought initially. Isn't this obvious, and rather basic?