News for hackers. Stuff that matters.
Re:Confirmed: Twitter switched temporarily to rate limited mode by minimaxir on Saturday July 01, 2023 @02:53PM (#36553713)
Update from Elon: https://twitter.com/elonmusk/status/1675214274627530754

> Rate limits increasing soon to 8000 for verified, 800 for unverified & 400 for new unverified

lolwut. Quoting a post from Mastodon: https://mstdn.social/@maxkennerly/110640373859329500

> LOL this is not how you deal with "data scraping," this is how you deal with a catastrophic loss of system capacity.

> You limit data scraping by blocking things a human user couldn't do, like access a thousand posts a minute. This is aimed directly at reducing normal activity across the whole system.

Re:Confirmed: Twitter switched temporarily to rate limited mode by marginalia_nu on Saturday July 01, 2023 @03:27PM (#36554057)
> You limit data scraping by blocking things a human user couldn't do, like access a thousand posts a minute. This is aimed directly at reducing normal activity across the whole system.

This isn't accurate at all.

Standard operating procedure for large-scale scraping is to use a botnet, it's a sybil attack where you can have tens or even hundreds of thousands of IPs ticking away at 10 requests per minute or some such. Done correctly, it's nearly impossible to detect which nodes are complicit from their access patterns or user agents.

That said, rate limits work relatively poorly against this. A proof of work approach like HashCash might work though.

Re:Confirmed: Twitter switched temporarily to rate limited mode by CamperBob2 on Saturday July 01, 2023 @04:29PM (#36554801)
Wouldn't limiting the number of IP addresses that can log in as a single user avoid the botnet problem?

That said, someone mentioned that the login page alone involves 11 MB of data transfer, so maybe that is the problem they are confronted with.

Re:Confirmed: Twitter switched temporarily to rate limited mode by tlogan on Saturday July 01, 2023 @04:38PM (#36554929)
The solution is not simple but there is one (but it is complex). The question is always are going to be are you going to spend time and money to implement the right solution or just pick up a cheaper one.

Could it be that they just stop paying bills for Akrose and they shut down their service?

Re:Confirmed: Twitter switched temporarily to rate limited mode by TillE on Saturday July 01, 2023 @03:16PM (#36553950)
I'm 50/50 on whether they're actually dealing with catastrophic technical issues, or if this is just Musk being mad about something and insisting on an incredibly silly solution. Both are entirely plausible!
Re:Confirmed: Twitter switched temporarily to rate limited mode by misnome on Saturday July 01, 2023 @03:29PM (#36554077)
I’ve seen speculation correlating “Going down at 9am on the first day of Q3” with the fact that they have apparently been trying to migrate off Google cloud by June 30th because of Melon’s blanket policy of “Not paying bills”.

Needless to say, if true, this would be extremely funny.

Re:Confirmed: Twitter switched temporarily to rate limited mode by joezydeco on Saturday July 01, 2023 @03:38PM (#36554199)
If Google decides to graveyard GCloud, you can say you were here when it started.
Re:Confirmed: Twitter switched temporarily to rate limited mode by jsheard on Saturday July 01, 2023 @03:07PM (#36553852)
Turning a big dial that says "rate limit" on it and constantly looking back at the audience for approval like a contestant on The Price Is Right
Re:Confirmed: Twitter switched temporarily to rate limited mode by ssnistfajen on Saturday July 01, 2023 @07:22PM (#36556389)
A garage startup doing this with a thousand users would be cool and fun. An established large platform with hundreds of millions of users doingg this is just undermining its own reputation.
Re:Confirmed: Twitter switched temporarily to rate limited mode by russdill on Saturday July 01, 2023 @03:15PM (#36553934)
I suspect this is less about scraping and more about artificial manipulation of view counts to both push certain scams/tweets to the top. I don't think they are providing advertising kickbacks yet, but that would only amplify this.
Re:Confirmed: Twitter switched temporarily to rate limited mode by rideontime on Saturday July 01, 2023 @03:02PM (#36553806)
So, like, two more feed refreshes per day? Watching the requests in dev tools, it's definitely based on fetches and not impressions.
Re:Confirmed: Twitter switched temporarily to rate limited mode by cududa on Saturday July 01, 2023 @03:00PM (#36553788)
A cool so I get about 3 more minutes of looking at tweets

My guess is he got rate limited himself after a few hours as a verified user so he’s already upping it

Re:Confirmed: Twitter switched temporarily to rate limited mode by suzzer99 on Saturday July 01, 2023 @03:21PM (#36553999)
They probably have if (isElon) somewhere in the code to turn off rate limiting.
Re:Confirmed: Twitter switched temporarily to rate limited mode by cududa on Saturday July 01, 2023 @04:15PM (#36554629)
Bold assumption that anyone left there would know how to implement something like that
Re:Confirmed: Twitter switched temporarily to rate limited mode by Kye on Saturday July 01, 2023 @07:43PM (#36556517)
Boolean was banned for sounding too much like boo elon.
Re:Confirmed: Twitter switched temporarily to rate limited mode by reaperducer on Saturday July 01, 2023 @03:34PM (#36554145)
Update from Elon: https://twitter.com/elonmusk/status/1675214274627530754 > Rate limits increasing soon to 8000 for verified, 800 for unverified & 400 for new unverified

Well, at least he's engaged over a holiday weekend.

If it was my company, this would be a fifteen meetings, a six-month series of sprints, and we still would only decide on a new color for the icon.

Re:Confirmed: Twitter switched temporarily to rate limited mode by soneca on Saturday July 01, 2023 @03:43PM (#36554259)
I guess a couple of well-run meetings would have prevented this disaster in the first place.
Re:Confirmed: Twitter switched temporarily to rate limited mode by jakelazaroff on Saturday July 01, 2023 @05:00PM (#36555156)
Six months of bureaucratic paralysis would also be better than actually rolling this out.
Re:Confirmed: Twitter switched temporarily to rate limited mode by ssnistfajen on Saturday July 01, 2023 @07:24PM (#36556401)
Sometimes things move slowly for good reason. Wasted meeting minutes is a reasonable tradeoff over rendering the main product unusable like this.
Re:Confirmed: Twitter switched temporarily to rate limited mode by evolve2k on Saturday July 01, 2023 @05:55PM (#36555696)
Some are suggesting that the rise in scraping is a direct result from Twitter killing it’s free API, and that there are a range of organisations whos work relies on this information who have now had to resort to other means of attempting to get the same info.
Re:Confirmed: Twitter switched temporarily to rate limited mode by gmerc on Sunday July 02, 2023 @08:16AM (#36560748)
There is no actual evidence of any of that though.
Re:Confirmed: Twitter switched temporarily to rate limited mode by l33tc0de on Sunday July 02, 2023 @05:24AM (#36559794)
Alphabet people talking about Twitter on Mastodon