News for hackers. Stuff that matters.
Re:Reddit is removing moderators that protest by taking thei... by thrdbndndn on Friday June 16, 2023 @01:03AM (#36351555)
Something I want to ask before:

I've noticed that some people have made a connection between the decision to effectively shut down Reddit's API and the increasing popularity of LLM/ChatGPT, and it does seem plausible.

What's interesting is that Twitter also closed down their API access merely months ago. Are these two occurrences related, or was Twitter's action also a response to ChatGPT/LLM?

(Also to be totally fair to them, Twitter and Reddit used to have the most comprehensive free API accesses which I enjoy. Instagram/facebook etc. are always a nightmare to work with in term of scraping.)

Re:Reddit is removing moderators that protest by taking thei... by SirSourdough on Friday June 16, 2023 @01:14AM (#36351659)
I believe this is more than speculation. Pretty sure that someone from Reddit basically said as much in one of the transcripts or conversations shared by Apollo’s dev. Reddit feels they are getting burned by letting their data be used to train LLMs for free.

That said, they were also letting people exceed their API rate limits by massive amounts, so hard for me to put the blame anywhere but with Reddit.

Re:Reddit is removing moderators that protest by taking thei... by lemming on Friday June 16, 2023 @01:17AM (#36351693)
Here's an interesting theory about that. I can't claim any credit for it, it's from Jim Rutt, I heard him on the Clearer Thinking podcast (https://podcast.clearerthinking.org/episode/160/jim-rutt-lar...).

He basically argues that LLMs are going to mean that vast quantities of the content on the internet are going to become automatically-generated drivel which no-one wants to actually read. I think this is pretty plausible. So in order to consume information from the internet, it will basically be a requirement to use an AI tool which, for a given query that you might have, will go out and retrieve all the information on it from many different sources, and return them, appropriately listed and summarised. Such a tool would probably want API access to the likes of Reddit, but would also make the main value prop of the way social media works currently (getting ads in front of eyeballs) useless. So end users will probably end up having to pay for their API use so that their tools can access these sites, and as a pleasant side effect they'll see no ads. So perhaps we'll end up at the promised utopia where users pay a little bit for the content that they like, and don't see ads along the way, as a result of AI. I have no idea how likely this is, but it sounds pretty good to me if it does work out like that.

Re:Reddit is removing moderators that protest by taking thei... by b800h on Friday June 16, 2023 @01:48AM (#36351956)
And then they'll put ads in the LLM chat stream. "Using texts on neurolinguistic programming and mentalism, find subliminal ways of promoting {product}".
Re:Reddit is removing moderators that protest by taking thei... by Accujack on Friday June 16, 2023 @01:14AM (#36351657)
>some people have made a connection between the decision to effectively shut down Reddit's API and the increasing popularity of LLM/ChatGPT

Those people really don't have an idea how either of those work, nor how cause and effect work.

See also the graph showing that the decrease in the number of pirates in the world's oceans has contributed to climate change because "pirates are cool".

Re:Reddit is removing moderators that protest by taking thei... by hn_throwaway_99 on Friday June 16, 2023 @01:09AM (#36351614)
That weird "conspiracy theory" doesn't make any sense. LLMs are trained off a static dump of example data, and it's trivially easy to build a webscraper of Reddit's historical content.
Re:Reddit is removing moderators that protest by taking thei... by buildbot on Friday June 16, 2023 @02:03AM (#36352084)
It's trivially easy to build a scraper for the new content too. And I think people really over estimate the relative amount of data reddit has compared to the literal rest of the internet. Like sure, it's nicely categorized and ranked (somewhat), but leaving reddit out of a corpus probably won't drastically effect your final model.
Re:Reddit is removing moderators that protest by taking thei... by boomboomsubban on Friday June 16, 2023 @01:46AM (#36351935)
The CEO is explicitly saying this change has to do with the ChatGPT's and such.

>But with artificial intelligence-powered large language models like Microsoft-backed ChatGPT and Google's Bard, a massive corpus of conversations is being hoovered up. And in return, Reddit receives very little, he said.

https://www.npr.org/2023/06/15/1182457366/reddit-ceo-steve-h...

And this isn't the first time they've said this.

Re:Reddit is removing moderators that protest by taking thei... by ttepasse on Friday June 16, 2023 @04:07AM (#36353097)
They could simply discriminate against LLMs via their APIs terms of service. That wouldn’t get bad actors, those would simply scrape. But that would be the case with the very expensive API.