Keep in mind that Twitter engineers are under extreme pressure, knowing that their H1B visas are at stake, to implement Elon’s whims as quickly as possible on systems they’re unfamiliar with. I doubt many of us would perform better in their situation.

Do you know how many current Twitter engineers are on H1B? Is this based on some data, or just speculation?

Around 300. https://www.vice.com/en/article/n7z5px/twitter-employees-on-...

That's all employees, not just engineers.

Do you think twitter has a lot of marketers, PR, product managers under H1B? I don't have data, but in the tech companies I have worked at it engineers were the H1B visa holders, almost exclusively.

They absolutely have a number of PMs on work visas. I'm friends with a lot of them.

I would imagine the vast majority of those are engineers. It’s pretty hard to get an H1B in a non technical field and I’m not sure Twitter would even have many of those types of jobs.

H1-B is for "highly skilled" workers, what skilled workers would Twitter have other than developers?

"fashion models of distinguished merit and ability"

https://www.dol.gov/agencies/whd/immigration/h1b

Potential trophy wives of billionaires, of course.

Is the assertion here that Twitter is employing H1-B fashion models?

You can look at the NAICS codes, the overwhelming majority are 54.

So maybe 20-30% of dev, which would be overrepresented here?

I think that may be outdated. Searching at https://www.uscis.gov/tools/reports-and-studies/h-1b-employe... yields around 60 applications in 2023 (but note that I'm not sure how things are counted given that H-1B is valid for 3 years).

The year is also only half over.

IIRC the H-1B application period is some time in the first half of April every year (so all the ones that would be filed this year have already been filed), but IDK if renewals need to catch that train.

Edit: clarifications.

H-1B approval has two stages, the registration period (March/April) and the petition period (ends 90 days after post-registration selection). AIUI this is counting petitions (i.e. I-129s), so won't be accurate until at least 90 days after selection notices - which would be around now - but then the site itself says it can further trail by a quarter.

You can access H1B job titles and salaries. It's public.

Yeah, I'd like some source on that, or I'll just assume you're pulling shit out of your ass because you jumped on the Elon hate bandwagon when Reddit told you to.

> when Reddit told you to

Is this projection?

Nope, but apparenly I hit a nerve.

I tried to point this out below. It's not even a DDoS, it's fake news.

https://news.ycombinator.com/item?id=36561808

it's a 429 error, so the developer who posted this is an idiot. they're not even wrong. the 429 doesn't even touch twitter's infrastructure. the HTTPS TLS terminates at a Google VM, which gets relayed depending on the VIP used to hit it, but the traffic never gets past that Google VM. This is literally /HOW/ companies deal with DDOS.

https://blog.twitter.com/engineering/en_us/topics/infrastruc...

it's unbelievable that my median salary for the past decade is $3,000 after several years of "Startup" followed by "How to Start a Reboot of My Life"

> If this is indeed due to the recent decision to enforce authentication for all API calls, it means the curlprit may actually be the API gateway or something similar downstream.

The way I understand it, DDoS is not caused by enforced authentication - enforced authentication is just a temporary measure against DDoS.

I would guess the front end was written under the assumption that the back end would still work without auth. Perhaps the backend changes (mandatory auth + rate limiting) were pushed without sufficient testing of the front + back?

Did Elon pay the AWS bill? That seems like a likely culprit. Twitter instances are being forcibly shutdown.

Twitter operates its own datacenters.

Yeah but they use GCS for auth, moderation, and caching. They apparently haven’t been paying Google since April and the contract expired June 30th

For someone that portrays they know so much about the agreements and deadlines you missed this article. It doesn’t help your narrative.

https://www.bloomberg.com/news/articles/2023-06-21/twitter-r...

Is using the word “narrative” the new hot thing we’re doing to try to discredit someone we disagree with? It’s so difficult to keep track.

It's been hot for a while now. "Narrative" and "agenda" / "pushing an agenda."

"Twitter and AWS signed a five-and-a-half-year contract in 2020, which AWS is not willing to renegotiate."

https://gritdaily.com/twitter-owes-aws-millions/

Twitter.com and the associated user-facing services do not run on AWS.

While it looks like they never started the move over to AWS, the press release makes it sound like they do use some AWS services.

> In addition, Twitter will continue to use AWS services such as Amazon CloudFront (AWS’s fast content delivery network service that securely delivers data, videos, applications, and APIs with low latency and high transfer speeds to customers globally) and Amazon DynamoDB (AWS’s key-value database that delivers single-digit millisecond performance at any scale).

I worked there. Services running on GCP are a significant part of the internal service infra (ml platform, etc.) and it's no impossible that the abrupt loss of GCP would cause user-facing problems. The GCP spend was many, many times the AWS spend. Unless things changed since last November, AWS is not a meaningful part of the internal or user-facing infra.

With respect to DynamoDB specifically, Twitter has its own custom distributed key-value store: https://blog.twitter.com/engineering/en_us/a/2014/manhattan-... that twitter.com itself runs on.

Thanks for weighing in with some actual first-hand knowledge. It is appreciated.

The latest on cloud hosting is from a week ago, and I'm guessing you don't have any more recent info than this:

https://www.reuters.com/technology/twitter-resumes-paying-go...

Correct, no more recent (or less public) info than that. Like I say, losing GCP could cause problems users notice, but sounds like that’s not going to happen.

And yet they also host with AWS, Google Cloud and Oracle. Cloud people take note: this is what lock-in looks like, and it's coming to a place near you.

No, this has nothing to do with lock-in. This has done nothing to do with decision making that subverts good engineering.

Cloud agnostic is hard

Well, they haven't paid their GCP bill... https://theconversation.com/twitter-is-refusing-to-pay-googl...

They’ve came to agreement.

Try to keep up with the story before you post 2 + week old articles that only show your bias. Thanks.

https://www.bloomberg.com/news/articles/2023-06-21/twitter-r...

HN doesn't seem to care about facts anymore, just "SpaceMan Bad!"

https://www.reuters.com/technology/twitter-resumes-paying-go...

I mean, when spaceman has been bad over and over and over you can’t be all that surprised when people see something go wrong and say “probably spaceman bad”.

Refused paying once is enough to say the man is stupid.

That's slightly outdated information:

https://www.engadget.com/twitter-has-supposedly-started-payi...

This is not remotely a likely culprit.

"curlprit" for too many GET's causing a 429 is just the perfect typo.

Could someone report the error at [email protected] and see what they think about it?

Quite the knee slapper. Thanks. I hope you didn’t spend many brain cycles on it.

Making jokes about Twitter is too easy.

Why do you take people criticizing Elon Musk so personally?

Do you think it's more or less funny than auto-replying to all PR enquiries with poo, when you are an incredibly large company?

Whoever is in charge of that account went all Oregon Trail on things and caught dysentery

While exponential backoff is theoretically optimal, I doubt it's actually used that often in practice. I've seen too many cases where someone decides serving user requests with low latency is so important that they'd rather have a constant randomized backoff than exponential backoff. I've been in many design meetings and seen enough documents where the decision not to use exponential backoff is explicitly made, understanding the tradeoff with overloading and system recovery.

I wouldn’t be surprised if this has no back off or limit at all. 10 RPS is fast enough that it may simply be sequential.

I’ve had to… uhh… eagerly advocate for exponential backoff for weeks of constant uptime issues before someone listened and actually implemented it and solved the problems.

Like several times in different roles.

People do it, exponential backoff is everywhere in your stack, but it doesn’t end up in your application layer until you have enough traffic that you actually have to manage throughout.