The singularity will come when the set of training data available for scrape is dominated by AI generated content and the AI's learnings are derivatives of what old AI's produced.

Human thought on the other hand has some sort of undefinable entropic-value that AI to-date is missing, a Human can produce a "good idea", whereas an AI produces a bunch of potential continuations of a stings of text and selects randomly amongst them (or, even better, a Human selects from them).

Unfortunately the advertising game mixes up the incentives and flips the equation so that the purpose of communication isn't to share a "good idea" as efficiently as possible, but rather to keep eyeballs on your website for as long as possible in the hope some flashy banner ad will distract your user and you'll get your $0.02 for them abandoning your page, likely unfinished. AI will (and already does) excel at this sort of task, but it's the kind of task that ought to have no value whatsoever.

Luckily we have increasingly sophisticated summarization-AI to go from the filler-AI generated crap back down to a couple of bullet points, but at that point you've invested millions of dollars, researcher-hours, engineer-hours, compute-hours, etc, to make the worst text-compression utility of all time.

By the way, it's not just text anymore, you can train GPT-3 on Youtube. You get aligned visual and speech channels and data in vast quantities. That means it's easier to tell if the source is human.

Bingo

It’s increasingly difficult to find product reviews with search engines.

Massive auto generated content farms take a product name and add loads of AI-generated filler text. Pop in a bunch of banner ads and an affiliate link and they have huge economic incentive to scale these operations.

I’m very pessimistic about the direction the internet is going these days. The AI crisis isn’t going to be sentient AI trying to kill us, it’s going to be a flood of noise over knowledge.

Sometimes it works to add "reddit" to your search to find interesting comments. I suppose that will eventually be gamed too.

Until we have to start making AIs to identify knowledge and filter out noise. And then a whole cat-and-mouse game between fake news AI and fake news detection AI.

This is the exact situation we are currently in.

https://rowanzellers.com/grover/

The end result is more robust AI and an understanding of failure modes.

I recall in the mid/late 2000's implementing a markov text generator to create thousands of static html pages based on certain keywords. This has been a problem for over a decade and will probably get worse as text generation tools improve, e.g. GPT-3.

I've got some insight into this (Several friends, now multi-millionaires ran and flipped tens of sites like this) They're mostly written by low paid content writers in 3rd world countries such as Vietnam and the Philippines. Primarily to drive affiliate traffic to Amazon and other retailers with affiliate schemes. They all operate on a similar format - 10 items with good reviews, write 300 words about each product, rinse, repeat, profit.

What a world where people can become multi-millionaires in this way while nurses and teachers can't even get cost of living adjustments.

Right now most of those pages are produced by much simpler models, which copy an existing page and replace text snippets with synonyms. I am sure that soon the spammers will switch to better models.

Had similar experience recently and it had made all the way to be the top Google news hit - apparently the site is cranking out ”news” as SEO spam to promote their app.

https://mobile.twitter.com/moo9000/status/145873329934659174...

been playing with copilot lately and even it just seems more annoying then it is helpful so far. Will continue experimenting but so far my impression has soured a bit

I tried Copilot and removed it from my IDE. Not good enough, only fills in obvious parts and even there makes mistakes.

I often wonder about this with Twitter accounts. How many are already GPT-3 generated?

We'll need another GPT-3 bot to detect the GPT-3 bots.