As I am writing, my iPhone 15 Pro is still accepting certificates signed by these compromised CAs. The device is "up to date", yet is still running Trust Store Version 2026073000. When I check at [2], certificates issued by 3 eMudhra Intermediate CAs are still being trusted!

Google Chrome revoked these certificates in 1 hour 40 minutes, Mozilla in just under 4 hours.

I am so astonished by how much Apple has failed here. There are potentially huge swathes of people completely oblivious that their phones are susceptible right now to being man-in-the-middle attacked.

Why does the iPhone not utilise a CRL list? Why does it take Apple 5+ days to do what Chrome did in under 2 hours.

It's a similar story for my MacBook, but at least there my Chrome browser _does_ use a CRL (the same can't be said for all the programs installed, however).

[1] https://fenko.nz/blog/emudhra-ca-compromise/#what-happened-i...

[2] https://emudhra.we-have-a-problem.com/

So actually 3 days ago.

Report OE11044088569486, but currently it's just under review and invisible.

... 5 days later and I know better than to do online banking or messaging on my iPhone 15.

https://news.ycombinator.com/item?id=50189521

I suspect Apple published no update and is relying on the CA's CRL and/or OCSP to do its job. But it's rate-limited, the connections time out, and iOS is hard-coded to soft-fail.

Like you my trust store is months old.

That said, I found separate installs of Firefox and Microsoft Edge that accepted *all* the bad certificates without a care.

The only browser that consistently rejected all of them was — you guessed it — Internet Explorer. A browser so old it couldn't even run the test properly but it did what it needed to and blocked all the servers.

I am not smart enough to say how, but the sheer amount of work this created across the industry is monstrous. It probably has cost $billions to mitigate, and those mitigations have been slow and ineffective (see: my iPhone still being vulnerable).

In my experience, seemingly random pieces of software, libraries, and programming languages packaging their own CA stores has made this much harder to mitigate than simply running 'apt-get update ca-certificates'. We've practically just had to update every single thing to mitigate this.

(Even with all the related options disabled, e.g., ocspd on OSX still regularly connects to random certificate related services to "check" on things. Something related to random root certificates I don't have in the system trust store or such, Not entirely sure what this is about and how to get rid of the requests in any other way than blocking things with Little Snitch)

It should be very easy and normal to ask for approval which is limited to the particular app, limited in time, and limited in scope.

Still keep defaults, but make it much smaller and put them under even higher scrutiny, certainly not 100 root certificates.

I don't understand why e.g. a CA for research institutions should be able to issue certificate for mybank.com or ecommerce.com and my browser would just trust that.

I am and the test site says that Firefox is correctly rejecting them.

Just now on M1 Pro Safari.

On M1 Prp Firefox: none.

On iPhone 17: Safari: none, Firefox: none.

I just tried on Safari, it still trusted 2/14 compromised chains.

Rubbish.

edit: I went to keychain access app and found all emudhar and em signer certs -> there were four -> clicked get info for each -> under Trust section of the pop-ip -> selected "When using this certificate: never trust".

Now all checks on https://emudhra.we-have-a-problem.com pass for me.

Scrutiny is inversely proportional to bit complexity of the private keys they used. Without a failure to apologize or expose how it happened: a human mistake, process mistake, etc. this looks pretty bad on them.

> eMudhra’s global SSL/TLS infrastructure runs separately from its other operations. The company’s Global digital signature certificate (DSC), eSign, and other signing services, including those in India, its enterprise PKI Solutions (emCA, CertiNext platforms), emSigner eSignature Suite, and its identity and authentication solutions (SecurePass, emAS) are not affected and continue to operate as normal

Anyone relying on them with anything related to security or trust is playing Russian roulette at this point.

Only when fully satisfied they'd have given license to operate this security business.

Companies already control all the traffic flow on their infrastructure using end-point security solutions, the same can be extended at ISP level, and then Country level

Typically they do this for non pay but they've dropped the hammer multiple other times incorrectly (and good luck getting a RFO out of anyone there).

It’s get worse when these companies work with regulators to create more regulation that they can sell more of their services.

https://www.amazon.co.uk/Compliance-Industrial-Complex-Opera...

Waiting for a release like "... the level of social media amplification around this incident has brought scrutiny that is out of proportion to OTP 1FA place in our business. In view of this, after detailed deliberations our board has concluded that withdrawing from global OTP 1FA is the right decision for eMudhra and its shareholders"

At what point does it become Russian roulette with a Glock?

> fear not, here's a complete bypass of their 1fa otp login system. they encrypt OTPs to a public key and issue those behind cloudflare turnstile. this key to be precise:

[1] https://www.youtube.com/watch?v=hXUqwuzcGeU

As a consequence, these businesses are very well protected from normal market forces and are pretty much above the law. Had this incident not been an international one, they had a very good chance of getting away with it, and the people reporting this would have landed in jail.

Would you say the system forces people to be like that or people are the reason why the system is like that?

Would you do business (i.e. trust) with someone that comes from that system?

It's the courts. Courts are meant to bring in an error correction mechanism. When courts fail, the country goes into open-loop control. No errors are corrected, and eventually, people stop even detecting errors. https://x.com/captgouda24/status/2110089855380369511

> Would you do business (i.e. trust) with someone that comes from that system?

Vet rigorously and continuously! Start with small business, and grow it based on interpersonal trust. Since the system lacks accountability, you must ensure that yourself.

In regards to doing business - someone that comes from a corrupt society with a culture of taking advantage of others should be approached more carefully than someone that grew up in the same town as you, of course

This is a lesson learned the hard way, where I moved to a different country and treated everyone with openness and generosity (of time, resources, willingness to help), just to realize people from certain cultures mistake this behavior for weakness and try to take advantage as much as possible

Anyway, the main point remains, not all cultures have the same values and if someone comes from a culture where taking advantage of others is celebrated, (for your own sanity) beware

Generally true, with a nuance. The root cause is often people, specifically its people with low IQ. When a society is corrupt you will notice that the mean IQ is lower.

Source: work on healthcare IT projects and can't let anyone outside of the US look at prod data no matter what due to the contract we have with our clients.

The government _mandates_ that certain services be used. In which world does this rhyme with "capitalism"?

The govt mandated services are owned by private capital(hence the capitalism part) and govt picks winners by granting licenses to a select few(hence the crony prefix)

When working with different businesses across the globe, I’ve noticed Indian correspondence to incidents tends to be more a show of strength even if it’s transparently obvious that there’s no merit to the content.

Whereas western European correspondence is often more apologetic and empathetic. Even if it might sound insincere to those affected.

But it’s all just words at the end of the day. It never means much unless the business is willing to back real change to mitigate similar issues in the future and resolve undisclosed existing problems. However there’s no way for us to know if this is happening or not.

"We admit no wrongdoing. We are only settling to move past this. Our business is strong/secure/competent."

It's like the first rule you learn when you go into corporate communications is to never admit you did anything wrong or project anything other than confidence and correctness.

It's the first rule you learn after joining basically any organisation larger than 5 people.

People on HN who are surprised are seemingly self-employed prodigies who dropped out of middle school.

Just take a look at any of the thousands of postmortems published over the last decade and you’ll see that the tone of incident response communication can vary massively.

On Mozilla’s tracker, eMudhra said the keys were created on 30 April 2025 in a key generation ceremony, a recorded procedure in which keys are produced inside hardware security modules (HSMs), tamper-resistant devices built to hold keys. The keys were meant to be moved to data centres in the United States and Europe for customers needing data stored in those regions. To move them, the company added a step to ‘wrap’ each key, or encrypt it for transfer, and ‘unwrap’ it at the other end. eMudhra said a flaw in the software script created a second, weaker set of keys during this step, and because the two sets had similar names, staff used the weaker ones by mistake. It said human errors occurred at multiple steps.

source:

What's the eMudhra security scare that has rattled the internet?

https://theprint.in/tech/whats-the-emudhra-security-scare-th...

If they go to the trouble of wrapping them, use real keys. Or don't do it.

Also, it's a big "human error" to not notice this for a year and a half.

It is more likely they had a script which placed a parameter in the wrong field, due to something like using positional arguments. Or they misread the documentation for some interface.

Still incompetence. And I would love to see my theory confirmed or refuted.

If you didn't know better, you'd think this had been done on purpose by some three-letter agency.

To be clearer, I’m not sure these keys were generated, but being able to use weak keys is something you may want to do for testing.

Code assumed an array of strings, instead got an array of bytes and picked the first one.

This is not a problem of devices and interfaces. You are supposed to generate the wrapping key on the HSM itself. And the HSM ensures that the keys are generated with a strong enough security parameter, not some silly 1 byte string. Then the HSM wraps the key inside of itself and exports it in the encrypted form. You take this ciphertext, and the wrapping key and send it over to the other party.

So if they generate the wrapping key with a funny value, that means they didn't use HSM to generate it but their system openssl. But then, HSMs typically don't allow you to export private keys without wrapping them first unless you first tweak around with security flags (not recommended at all), or simply generate the private key using your system openssl too (LMFAO).

Source: I work for one of the major HSM vendors.

Generating a key with "4" was probably done as a joke, but also is insane to have ever lived outside some devs computer.

Oh come on. They clearly didn't intend to use 4 as a private key or to wrap it. They had just purchased millions of dollars of HSM's to store them. It was something more subtle than that.

Have a little imagination. The transfer procedure sounds bespoke - possibly just a shell script using openssl to encrypt things at one end, and decrypt them at the other. Let the man who has written 100 shell scripts without a bug in one cast the first stone.

But not so subtle that good software engineering procedures wouldn't have picked it up. In fact today just an LLM review of the shell script probably would have caught it. The Internet Death Sentence was warranted - because poor software engineering procedures are unforgivable in a CA. But those procedures and policies aren't easy or cheap to come by. If they were just "don't chose 4 as a primary key" my grandkids could do it.

and imagine being responsible for halving your corp's stock

Unsurprising if you are familiar with the indian honor code „izzat“. To quote an old kiwifarms thread:

Hi. I didn't see this term mentioned in the thread, so I wanted to share this all with you. It's about Izzat. Some posts have mentioned 'saving face', which is Izzat, but I wanted to elaborate on it.

Izzat has no direct translation into English. We only have terms that can broach the same concept such as 'honour' or 'reputation' or 'face'. Izzat is so much more than that. It's a zero-sum game of collective honour shared by whole groups of people, all of whom take it very, very seriously. A system like this isn't just foreign to Enlightenment values, but I'd argue it's antithetical to every sensible form of governance on the planet. It will destroy any system that assumes good faith.

Izzat conflicts are not about who is right and who is wrong. It's about who wins and who loses. This means it's a zero-sum game where just about any action is justified (including murder) to restore the lost Izzat. Izzat is a limited social currency and the easiest way to get it is to take it from someone else. Winning is righteous in Izzat. Losing is unrighteous. This means that if someone plays the game of izzat well enough, they can get away with just about anything (murder, rape, scamming, cheating, stealing). The only morality in Izzat is the protection of your group's collective ego. The only appropriate response when your Izzat is attacked is the complete destruction of whoever insulted it.

Two people get into an argument. They might escalate, but chances are it won't be that bad. i.e. 'Sorry I broke your garden gnome.' With two Indians in an argument, the stakes are always deadly thanks to Izzat. Neither of them can back down, nor can they admit fault. Admitting fault is seen as deliberately humiliating yourself. Not only that, but because Izzat is shared, you are shaming everyone who shares your Izzat. So, admitting fault or taking responsibility for a problem is a form of social suicide. This means even if the dispute was over something completely fucking stupid or trivial (like a broken garden gnome), it could spiral into a decades long honour feud. Izzat is also the reason why police are reluctant to get involved in disputes. Because Izzat is a zero-sum game, all participants are involved in the game. If a policeman sides with one family over another in an Izzat dispute, this means that he's deliberately taking the side of that family and dishonouring the other. This marks him for retribution by the offending party. Without parties being able to be impartial, then centralized authority cannot effectively function.

Let's say someone in India complains about a broken water pipe. Instead of the problem being addressed, the official responsible for the water pipe denies it's a problem and counterattacks him instead, because daring to question his efficacy in his role was challenging his Izzat. So the official destroys the person who brought the problem up. The water pipe never gets fixed.

Izzat is also the reason why Indian managers are so infamous for hiring more Indians. It's because from the manager's perspective, he's using his position to gain an invisible social currency. Merit and actual qualifications come second to that idea. If he hired a westerner, he would not gain or even lose Izzat by doing so. He has a very strong cultural incentive not to be impartial. Meanwhile, if he can strong-arm dozens of Indians into a company, he is gaining huge quantities of Izzat and conspirators who owe an absolute debt to him.

It's not uncommon to see Indians gloat about their success in the west. And yes, izzat is very much a system that enables short term success. But the fundamental reality is the prosperity that these Indians find so attractive in the first place wouldn't exist if the west practiced something similar to Izzat. Our systems can only exist on the assumption of good faith, and not a majority of people exploiting them for destructive short term gain. On top of that, if an Indian causes the systems and companies he comes into contact with to collapse, then he can just go back to India with his plunder. He has no stakes in the long-term prosperity, functionality or stability of these systems. The stakes are completely asymmetrical in the Indian's favour.

You all know about that infamous video of that Indian scamming a food bank. Once again, winning is righteous. Losing is not. By employing 'clever' means (jugaad), he successfully extracted more resources for himself and his family. From the perspective of jugaad, a rule is not something to respect, but merely an obstacle to the Indian's own gain. Since he cheated the system and wasn't caught, he is seen as a righteous and dignified man in the Izzat framework.

I read a story in India about a man offering to pay another man to use a public toilet instead of defecating openly. The latter left and came back with several friends and beat the former to death. The problem was not that he was defecating openly, but that he was criticised for doing so. And with Izzat, it's not an eye-for-an-eye. Izzat is often an eye-for-a-whole-head. Disproportionately and brutally annihilating your enemies is the correct move to make because you take back your lost Izzat and then some. This kind of vindictiveness would be rightly seen as horrifying and disgusting in most of the world.

If they expect people to still trust them on their other services they have obviously no clue on how Trust works, meaning they shouldn't be in the game in the frist place.

As a trust provider you get one chance. Squander it and you're out. That is how it should be.

In any legal dispute involving them, Exhibit A will be "look at how they ran their SSL/TLS business". That won't go well.

It would be like me practicing surgery 1 hour a week and saying "well surgery only has a very small place in my 80-hour work week" in response to flagrant failure of medical care.

Yeah, the proportion if anything is too small. This is the sort of company that should not be doing anything related to security.

The more telling thing to me is that they explain that THEIR core business is unaffected, but they don't explain that they fucked up so badly that everyone else had to step in and actively update their trust chains to work around eMudhra to avoid putting every single internet user at risk.

Quite frankly I doubt they understand how massively they've screwed up, or they are trying to hide it to avoid absorbing any blame for the fallout.

>According to eMudhra, proper strong keys were created in the HSM at the key ceremony on 30-04-2025. The script had been changed to test wrapping and unwrapping keys, because the intermediates were to be moved to data centres in the US and Europe, and a bug in that change created a second set of weak keys. The keys had almost identical names, and the weak ones were used for the certificate requests. The same request was signed by eMudhra's own root and sent to Sectigo, which cross-signed it. The strong keys never made it into any certificate (Bugzilla).

https://www.fairssl.se/en/articles/emudhra-ca-noegler-knaekk...

If transfer were to be allowed I'd think it should be entirely facilitated by the hardware itself and that capability audited as part of the approval process. (I assume the HSM used in this context is already subject to approval?)

Note that 32 0x00 is a possible output from a pure random generator while still being quite compressible, you would probably still want to throw that one out though. After buying a lottery ticket.

Maybe, like, setting up a factory that builds both LHD and RHD cars and building cars with two passenger seats in the front for two years straight without anyone noticing?

The king was very careful to ensure all the gatekeepers were following the process correctly. It did not occur to anybody that one of the gatekeepers could be using a dirty rag to jam their gate closed instead of a lock.

Once discovered, the wall could be trivially bypassed. The security of the whole internet was immediately broken. Rebuilding the wall to block up the dodgy gate took much effort from very many people.

This is why I went for a "wall with many gates" instead; any bypass makes all gates redundant.

All analogies break down at some level; the main goal is to clearly communicate the most important ideas.

The reason I agree with you is that it was too obvious i.e. if you were backdoooring something, you would want it to be more difficult to detect than this.

I guess the main takeaway is that the entire Web PKI is a far bigger house of cards than everyone assumes. Ignorance is bliss, I guess.

that is untrue

Citing Terry Pratchett’s “Interesting Times”.

This system was invented in the 90s when things were looking up. China was opening up to the world, the Russians were suddenly not evil anymore, and the internet was supposed to bring us all together into one happy global society, kumbaya and all that. Obviously that didn't work out.

It's time for a rethink really. Every CA is now a single point of failure.

At absolute minimum any sensible system would have a TOFU element whereby the site operator of example.com changing from one day to the next would result in all the tooling across the entire stack screaming loudly (in other words what ssh does). And that's just the absolute bare minimum change that's needed.

Normally, the king would declare and sign a proc stating that anything signed by his secretary's "unforgeable" signet was to be respected as well. The secretary could delegate authority to others in turn.

Well turns out that one of the signets in the chain of trust was just a picture of a "four" that anybody could have forged, but nobody noticed because in all of the ceremony and stacks of parchment with wax seals, nobody actually checked if the signet was secure.

However, Kings likely benefitted from the option of a scapegoat, they could always blame someone for forging their unforgeable seal and have them killed, rather than admit that the seal was true.

If we add in attainder and escheat, the King had a pretty strong hand.

As it turns out, these documents were mostly useful for long-lived décisions, which could be overturned, and where a forgery exposed you for a lifetime.

[1]: https://en.wikipedia.org/wiki/Robert_III_of_Artois#The_fraud...

- emudhra lawyer maybe (:

> "We set up a system where every government and a bunch of random banks and IT companies in the world holds the secret codes to everyone's encryption. Then we make them pinky swear not to abuse it, with stern talkings to if they do. And now it's all on fire, because one of them was irresponsible!"

You can know the intractability of the problem that PKI is solving, and the (worse) alternatives that have been tried, but that doesn't make this system not-insane to a normal person, and its at least a little bit the fault of the collective "us".

And it's not like it is an impossible problem; we rely on TOFU for SSH, we have remotely-attestable TPMs everywhere, ZKPPs / PAKEs exist, session keys and federation protocols are widespread. Somewhere in the set of "possible solutions" are options that don't sound unhinged when you explain how we got to this eMuhdra situation.

Then, everyone who trusts the CA will trust your certificate by checking it is signed by the CA who should have done the checks to ensure your certificate contains truthful data.

So your layman description of “they have secret codes to everyone’s encryption” is completely wrong.

A modern TLS connection uses an ephemeral key-exchange protocol where both parties opaquely agree to a one-time-use random encryption key that is not knowable to any other party, not even someone who can gain access to either party's private key. This is called Forward Secrecy (originally and sometimes still known as Perfect Forward Secrecy or PFS, but the use of the word "perfect" in any cryptographic context is akin to naming a ship "Unsinkable").

> random banks and IT companies in the world holds the secret codes to everyone's encryption.

That's not true at all.

What they do have is the power to MITM connections IF they first spook DNS or insert themselves into a proxy accepted by the server somehow.

So, if you want to be honest, the original sentence becomes:

> random banks and IT companies in the world holds secret codes that are trusted by everyone to sign information that is used to establish trusted communications. If they wanted to, they could try to tamper with people's network connections and, if successful somehow, pretend to be anyone else that had used their secret codes to sign their information.

Quite a big difference, no?

I find that communicating just enough to allow the formation of an roughly accurate threat model better retains attention; and in this particular situation I was being somewhat glib to emphasize the point of how insane our PKI system is.

> IF they first spook DNS

Or arp poison, or BGP hijack, or get in-path via tenancy shenanigans... TLS exists, premised on the literal assumption that such things are possible. If your argument is that we don't need to worry about a TLS weakness because such things are hard, I'd ask why we even bother then. CA compromise is the literal worst-case in PKI-world.

The certs are used to validate website identity, whereupon key agreement is performed to establish a symmetric key that has nothing whatsoever to do with your certificate.

All that matters here-- all that we use the keypair for-- is to prove that each party "is who they say they are", and that is something your CA can lie about.

If a layperson understands,

    able to issue arbitrary certificates that an arbitrary browser will trust 

    has the secret codes to everyone’s encryption

  for the non technical folks:  imagine there was a password that if you guess   it, you can impersonate *any website on the internet*
  
  now imagine the password is literally the number 4

Except it's still not good enough, because you could use that key to eavesdrop nearly anything and anyone.

As a 90s kid, the Dodge Omni was a common shitbox student car. I think they had like 8 key patterns lol.

My brother’s ignition key unlocked my friends’s trunk. There were a few people who could start other peoples cars. In one case, a soccer player “stole” a football players unlocked car as they looked identical and were parked near each other. In another as a prank we unlocked cars to roll them out of parking spots and physically lift them into unusual places.

I start by saying this is about the tech behind the S in HTTPS. I say that the main point of this is that when you talk to foo.com, it has to prove that the people behind foo.com are on the other end, so it's safe to send them your encrypted password or whatever.

Then I say a bit about how the system works by having a small number of trustworthy companies vouch for other people's servers. This doesn't need too much technical detail to get the point across.

Then I say one of those "trustworthy" companies put an encryption key of 4, where it's supposed to be a long number that's hard to crack. This makes the verification easy to forge.

Then I say that everybody who runs a server needs to remove this company's bad key, and Windows or Mac or your phone needs updates too. And that your web browser probably already quietly removed it, which removes a lot of the risk.

I feel like when I explain it this way people get it.

Hey the 4 was picked for good reason!

xkcd.com/221/

Mine was the lock company selling locks but also making freely available keys that would open any of their locks.

1. 4.7 per 100k people per year vs 288.5: https://en.wikipedia.org/wiki/Motor_vehicle_theft#Motor_vehi...

If you measure auto theft rate per vehicle (not per person), then the US has the same rate as Canada. There are 16% more auto thefts per capita in the US, but the US also has 17% more cars per capita than Canada.

If lack of immobilizers was the main factor, Canada would have significantly lower auto theft rates than the US, and countries that mandated immobilizers would not have theft rates 15x higher than countries where they were uncommon. Neither of these are true, so immobilizers are most likely not the main factor in auto theft rates.

Auto theft rates are mostly determined by car ownership rates and the number of auto thieves on the street. The number of auto thieves on the street is determined by how quickly they are caught, and how long they are locked up for. Auto theft recidivism rates are higher than any other crime, including drug offenses. Around 80% are re-arrested within 3 years of being released. Considering the clearance rate for auto theft is around 10%, that means the average auto thief is stealing a lot of cars before being caught again, and the actual recidivism rate is higher than 80%.

And I'm sure Hyundai and kia stats would be much higher due to lack of immobiliser and visibility of how easy they are to steal on social media.

1. 66,625 vehicle thefts divided by a population of 27,600,000. https://www.abs.gov.au/statistics/people/crime-and-justice/r...

2. 659,880 vehicle thefts divided by a population of 341,784,857 is 193 per 100k. https://www.nicb.org/news/news-releases/us-vehicle-thefts-ex...

Then the intern decides, “I can just glue these identical little white circles with blue sparkles onto the screw, and it will be faster and I can take more breaks!”

The best example I can think is (some) international borders: yeah, they're heavily regulated entities with lots of written protections (and thousands of miles of impenitrable barricades), but occassionally a hole opens up (whether physically or engineered).

Perhaps a section of border fence is occassionally destroyed (and people sneak in), but this won't happen if the border guards are approving [false] passports from a compromised government actor ("they have a valid passport, according to our inspection"), the border fence doesn't even matter!

Presume anybody can now go to that bad-passport-issuing government and request a passport to then enter any other country (i.e. the other country will see the passport as valid in the official global database): BAM, borders no longer mean anything; welcome to Privileged Accessia.

And when that clicks, extend the metaphor, literally, to a key with ~384 grooves.

“That's the stupidest combination I've ever heard in my life! That's the kinda thing an idiot would have on his luggage!”

house collapses and people get injured.

concrete manufacture says we no longer build houses cause it’s too small of a business.

I'm actually being serious, I do judge companies by their logo and website design taste, and more modern-looking companies also tend to pay more attention to doing things properly.

Then they locked it with the key you find on a chew toy for a baby.

Your computer has a built-in list of signatures that it trusts and expects to see on the certificates, and one of those signatures was eMudhra’s. These signatures are basically just a proof that you know a secret number without revealing the number itself. If you know the secret number, you can sign the certificate.

There are an unimaginable number of secret numbers. Imagine each atom in the Great Pyramid of Giza is a secret number. And then imagine you have as many Great Pyramids of Giza as there are atoms in the universe. And then double it.

Now imagine you choose any of the numbers represented by all those atoms to sign the certificate with, and you chose 4.

It’s not technically wrong: 4 is indeed a number. But if anyone wants to guess your number, and they start at 0, it won’t take them very long to find your secret number.

Once they’ve found your secret number, they can now sign certificates pretending to be you. So someone could pretend to be your bank, and present a certificate saying “yes, I am your bank” with a signature on it that your computer trusts.

eMudhra sent you a padlock that had the code "4815" promising no one has ever used it or would be able to guess it, but all anyone really needed to try to get in is set the first number to "4" and ignore the rest.

It's not entirely accurate, but its accurate enough for them to get.

Yours is much funnier!

https://youtu.be/a6iW-8xPw3k?is=IFYVVkYnlKQk1uh7

In order to sell something in the market, you need to prove that you are who you say you are. There are about 200 trusted clerks whose job it is to check your papers and verify your identity. Once they're satisfied, they stamp your papers with an intricate stamp designed by the finest cryptographers.

Anyone seeing that stamp knows that a trusted clerk has checked your identity and approved your papers. Most people don't examine the stamp closely. They simply recognize it as belonging to one of the trusted clerks and accept it.

One of these clerks, however, instead of commissioning an intricate stamp from a master cryptographer, bought an ordinary rubber stamp from an office supply store.

One day, a particularly paranoid cryptographer was examining the stamps used throughout the market and noticed something peculiar about one of them. It wasn't intricate at all. In fact, it was so simple that anybody with a few dollars and a trip to the office supply store could make an identical stamp.

The clerk's job wasn't merely to stamp papers. The clerk was supposed to check that everything written on them was true before applying the stamp.

But now anyone could write whatever they wanted on their own papers and stamp them. The resulting documents would look exactly like ones that had been carefully checked and approved by the clerk.

A robber could write "thebank.com" on their papers, stamp them with the office supply rubber stamp, and present themselves as thebank.com. If they could get a customer to come to their stall instead of the real bank's, the customer would have no reason to suspect anything was wrong.

After all, the papers had the stamp of a trusted clerk. And everyone who trusted that clerk's stamp would believe them.

  - you know how some people have passwords like 1234
  - ha ha yeah what idiots
  - what do you think an internet infrastructure company password is like
  - idk something long and hard to remember
  - 4
  - lolwat

Not 100% accurate, but conveys how stupid and incompetent all this is.

https://jeffwerner.ca/kryptonite_lock/

People swapped out their bike locks en masse, it was a pain.

Everyone agrees that passports aren't made by one company or one country, yet all countries accept passports.

Not everyone can just start making passports though, it's a complicated process of gaining trust and validation as a "real country" before you can issue passports.

One "country" started using a bit of paper that said "trust me bro" and everyone had to treat it as a valid passport. Worse, the way the internet works is that the border guards don't actually see who issued the passport, only that one of the trusted countries produced it.

Having passports this easy to forge means anyone can anyone.

It would need to be a 3, 5, or 7, since a prime is required.

Note that p,q (the primes you’re thinking of) are not actually required for decryption. They enable you to calculate d, the decryption exponent, and also enable decryption-time optimizations (exponentiation with d_p and d_q followed by CRT recombination), but that’s optional. In fact, p,q can be computed from n, e and d.

q=82392744353585332429460820191121154258120959554076291572887591391377445536703

n=6961473881062880999817189307002435362675293718841800458853975151946795609411283948376850761685158526939121778595276868395992041449207352173613540433589369

e=1305276352699290187465722995062956630501617572282837586035120340990024176764584449553695476096002966088824142338449519554315283866861919281309484272012046

d=4

assert n == p * q

assert pow(pow(123456789, e, n), d, n) == 123456789

(This doesn't work for all messages `m`, so I agree with what you've said - d=4 is not a good choice. But it can work.)

Every definition of RSA I've ever seen requires d and e to be coprime with λ(n)

This feels a bit like an EC25519 definition suggesting that "private keys must be significantly larger than 4" and therefore you call what eMudhra generated "modified EC25519"

Also note that using small numbers like 65537 for RSA public exponents is common practice.

Like, that had to be malicious right? ssh-keygen won't generate a key like this, and neither will openssl or whatever Microsoft is doing behind their CA server.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=2083449

shrug

I think they're just absolutely inept and should never be trusted with anything security related as an organisation if this is a mistake they could make.

The key needs to mention which curve it is using.

Let's take your words only.

Manual processes which are bug-prone and faulty are getting outsourced. They handle it with their best knowledge. Some processes/data knowledge is never transferred and some are kept hidden and assumed from outsourced teams but they have different culture and exposure so their understanding of things might be different. Its expectation mismatch not team issues.

American software giants are run by Indians only. So, if they are bug-prone & faulty then how could they reach there. It just that when person study with you or live with you then they have same set of culture & exposure like you have. So, they can manage things effectively.

Just noting an observation. These are not outsourcing agencies but GCCs of American and European companies. They had complete option to automate things or fix security defects. They simply didn't know what they were doing was wrong.

I don't mean to imply all Indians are incompetent. There was significant contribution from India to computer science. It's just that the ratio of competent to incompetent people is low. This is on your education and hiring systems.

https://en.wikipedia.org/wiki/Body_shopping

Again you seem more concerned with grammar than, y'know, condemning racism, prejudice, or bigotry. They're all equally bad and closely related concepts. "A rose by another name" and so on.

Your own comment has 3 typos and several grammatical errors ("prejudiced to" is wrong, for example). Irony.

Finally, modern usages of "racism" encapsulate all of these meanings. Languages change over time.

> You can be prejudiced to any group of people, doesnt mean that you are biggoted

Yes, that's literally what it means.

All that said, I make this comment as I believe the other commenter's worry is on display im this comment here...

> Again you seem more concerned with grammar than, y'know, condemning racism, prejudice, or bigotry. They're all equally bad and closely related concepts

These are _not_ all equally bad. Prejudice is foundational to understanding, there is no escape from this. In this understanding, we should treat particular postures of prejudice more serious than collapsing any "bad prejudice" under the same label.

Do you think racism is worse than bigotry? Why? I understand that the first word has a more emotional charge in our times. But as far as "bad things to do" go they're equal in gravity. Racism is just a subset of bigotry.

> These are _not_ all equally bad

Why not?

> Prejudice is foundational to understanding

That's one definition of the word - "judging without knowing the facts".

But there's another definition - "an irrational hostile attitude, fear, or hatred towards a particular group, race, or religion" - which was the one we are operating with in this thread.

https://en.wiktionary.org/wiki/prejudice#English

> I have these arguments with people I love who are right in spirit and still dig in the moment the word “racism” lands, because to them it names a thing they’re sure they aren’t

Like I said, it's because of how emotionally charged the word is. People think if they label unfairly discriminatory or prejudiced behavior as something other than racism, it's somehow not as bad as racist behavior.

They are wrong.

> Everyone in this sub-thread could easily agree it was out of line (and no one directly says otherwise).

I don't think everyone agreed it was out of line. There were only 2 comments calling zoobab's original comment racism. One of them spawned this thread about which words to use, rather than agreement with the condemnation.

The other comment https://news.ycombinator.com/item?id=50196949 that called it racist still looks gray and downvoted to me.

That's why I fight about stuff like this. It's all too easy to brush it under the "not racism, totally cool" carpet otherwise.

https://en.wikipedia.org/wiki/Endianness

I had a bunch of openwrt routers based on the ADM5120 SoC, some of them were of different endianness, totally wth:

https://linux-mips.org/wiki/Adm5120

But either way, I find the reduction of the argument to the semantics of the word racism to be deflection. Wouldn't you agree the original comment was made in poor taste? Why did it need to include a reference to Indian and imply that all things Indian made are cheap or low quality?

We applied this kind of mindset to the Japanese in the 60s and 70s, and look where that got us. I witness it happening again in the 00s and 10s applied to Korean electronics (LG and Samsung mostly) while we were content with minor improvements over the status quo and innovating far too slowly.

Yes, this incident has an almost farcical amount of stupid attached to it, but the fact that it happened to an Indian company is in my eyes only coincidental. There are plenty of Silicon Valley startups and tech companies that are equally cavalier about aspects of security and apply the mentality of "we'll fix it in post" to anything but shipping features.

Curious which language is that, and if "racism" is an accurate translation to English. What you describe sounds more like xenophobia.

> Racism is a form of discrimination in which a person is judged on the basis of his or her ethnic origin, skin color, nationality or religion. It is a form of prejudice and inequality that prevents people from gaining equal opportunities.

While I'm happy to admit that perhaps my use of the word in English is incorrect and its meaning there is more narrow, in Dutch it covers nationality or religion as well. Islamophobia is considered racist ("racistisch") in Dutch, for example.

[0]: https://www.nationaalwoordenboek.nl/woord/racisme.html

Most companies handle security very badly.

The "4" would be valid in hex, decimal, and even octal.

https://xkcd.com/221

I was hoping the statement at the bottom would be written in past tense: eMudhra was a global provider of digital identity ... had temporarily enabled secure digital transformation ... Its product suite spanned CertiNext...

This seems like a weird statement given that anyone can issue certificates out of the affected CAs.

The more interesting targets will be various REST etc apis being called over https. Lots of people doing API clients have been ignoring the "what if someone tricks me to connect to an evil server" vector, because they assume TLS verification will protect them.

They just didn't mention that now anyone can be their own Certificate Authority.

It's a godsend for those looking to do what they want with old hardware. You can now pretend to be the authoritative domain for updates.

Is 14 hours a diligent response time for something like this?

It was ~15hrs 30 minutes to first revocation

Good that we are now investing thousands of engineering hours globally to fix this.

Several tech stacks require upgrades of their runtime, python is an ungodly mess with certifi, but yes. It's just 1% of your revenue.

Sorry, it's been a long week.

It might even be sensible to intentionally revoke a small number of certificates every month, to make sure nobody is complacent.

My wife bought this fuckass IoT air purifier that's constantly phoning home. I'm gonna figure out what it's telling the mothership.

https://www.mitmproxy.org/posts/releases/mitmproxy-12.3/

My wife still plays her freemium games, the ads just don't load. She took out one of those games when we were sitting at the airport once. She was frustrated that her once-pleasant strategy game was suddenly crammed full of ads for boomer slot machine apps, and I had to explain that the app had probably always been that way.

I am reminded of the US DST changes back in the 2000s: there was all sorts of drama because a lot of software is written in the US, and there was much 'computerization' in the 1990s, but US coders had never personally experienced a DST rule change during that time period, so any change needed a reboot to pick up the new rules.

That drama led to (e.g.) periodic dynamic checking of /etc/localtime on various Unix-y system.

Sometimes you don't realize that things can be improved until you experience situations in which they currently suck.

> He has wide experience spanning over 45 years in Banking, Financial Services and Technology Industry. His profound understanding of the digital security landscape has been instrumental in eMudhra's global expansion and innovation. ...

> A thought leader and author, Mr. Srinivasan penned "New Age Management Philosophy from Ancient Indian Wisdom," ...

> Under his visionary leadership, eMudhra has become a cornerstone in digital security, known for its pioneering solutions and steadfast adherence to global standards.

https://emudhra.com/en-us/leadership

---

The CEO guy had 45 years of experience and still managed to make a mistake that would get a high school student an 'F' on their hw assignment.

I can even hear this conversation happening: what? I need to open this America/Los_Angeles file just to convert a timestamp? What pretentious overengineering! I want a single-file executable with zero dependencies. Let me hard code the rules.

Isn't this what Go programs are to a certain extent?

Actually Go allows you to embed the tz database instead of forcing you to hardcode, so this issue probably won’t be very widespread.

We've had to rework part of our application and certificate use because of this.

Let's call it "Let's Decrypt!"

Now we are in a situation where we don't really have a practical and correct way to revoke certificates, but many, many things depend on pki.

No, this is not even remotely sensible.

An analogy would be shutting down neighborhood streets versus shutting down major highways in a state for a hurricane evacuation. You can't test the latter at scale.

They'd be created with big red warning signs, installed into browsers and other places by anyone who wants to test, and then publicly revoked every month. Rinse, repeat.

The original comment was to do a fire drill and revoke live certificates to force everybody to participate in the test.

You can have a fire drill and evacuate an office building, but you can't hold a full-scale forest fire/volcano/hurricane drill and evacuate 3.5 million people to test mass evacuation routes, and also you got the edge cases that don't participate in drills, like homebound seniors or hospitals, where disaster strikes.

Do they, actually? I got the impression from the article that little revocation testing of any scale happens. If so, then even a limited test would be useful.

Code paths and procedures that aren't exercised regularly are broken.

Even better is having a way to reoke keys and big players occasionally include revoked keys in trust chains and blast a warning if the revoked key is considered valid.

If you don't know if you can hot-swap your certs, you probably can't.

But at the end of the day if you're not prepared to revoke the trust of any CA you trust, it's a you problem

"no one was quite prepared for intermediate certificate revocation" is actually worse because as an intermediate CA, they obviously didn't meet the requirements to be a full root CA, so why wouldn't they be more likely to need revocation than a root? Honestly more negligent than being unable to revoke a root.

That doesn't mean it's cheap or easy, or that it's reasonable for the responsible party to brush off all the havoc they've caused.

This demonstrates how hard we are messed up in a CA revocation. But no need to rub it in like that, is there?

I can see the case on phones: the OS may no longer be supported by the vendor, so browsers shipping their own set makes sense. And on embedded there isn't (and on IOT there might not be) an OS, but then you should probably just ship the the single CA that you use, not all of them.

There are reasons to do this, like support for minimal environments, consistent cross-platform behavior, simpler deployment, and application control over root certs.

So they didn't even use their own certificates? :)

i'm not sure how much i should read into this, but if you're sending out warnings to people susceptible to ask chatgpt about it, don't say the company is Indian.

But you know your audience better than ChatGPT does. Whether people react better, worse, or indifferently to the India angle will depend on the biases of the recipient.

Adding "out of the affected CA's" is top PR weasel wording that will slip by non-technical people who read this.

Congratulations to those who wrote or prompted this statement.

If this company was so incompetent that their private keys were a single byte, I seriously wonder what other areas they're incompetent in but that they've ventured into anyway. When you fail so spectacularly and so publicly at something like being a certificate authority, in the manner that eMudra did, it's rather hard not to wonder.

The private key of your globally trusted CA is infrastructure data and it was breached.

> All physical and logical security controls remained intact.

No, because you promised the CA Program that those keys would be HSM generated, FIPS 140-2 level 3 compliant. Ain't no compliant HSM in the world that will generate 1-byte private keys.

> There were no end entity certificates issued out of the affected CA's.

I charitably read this as "revoking these weak CA didn't affect customers". But now your whole root is being revoked next week (which you forgot to mention). All your TLS customers get screwed.

Overall, good choice to give up, but lying on your way out is not a good look for a company with the tagline "Trust Delivered".

As in, Chrome will distrust your CA and your business is dead. I think that's reasonable.

So far their responses to critical questions in the Bugzilla (https://bugzilla.mozilla.org/show_bug.cgi?id=2083449) have been rather weak and delayed.

https://uit.stanford.edu/service/ssl/emSign

they should tear down the entire barn

Popcorn recommended for the read!

lol, I didn't realize the latest comic was also topical.

https://xkcd.com/3314/

lol

It looks like the halfsix has a problem. Is gird fine for now?

What a weak minded statement. Just admit you messed up and call it a day.

The statement can use whatever weasel words it likes, but the action pretty clearly says "we aren't mature enough as an organisation to handle this responsibility." Which would be quite the concern for me, if I were a customer for one of eMudhra's other product lines.

It raises the question why these CAs agree to participate on this forum and then complain about doing so.

1: https://bugzilla.mozilla.org/show_bug.cgi?id=1950144

Folks in India — especially the boomer class — are trepidated by how they're perceived over actual substance and quality of their work (exceptions exist, of course).

Edit: Sudhir Kakar's (an Indian psychoanalyst) work on Indian identity (The Inner World, Intimate Relations) helped me decode my own origin culture's internal logic. Highly recommend it for people of South Asian diaspora.

In martial arts the winner will often say "I got lucky" or "you almost had me" to allow the loser to save face, at least publicly, when in reality the skill gap is evident.

A bad loser tries to save face. A good winner helps them.

The company we're discussing is being a bad loser.

no grace if you use 4 as the private key for a trusted authority.

> Izzat (Bengali: ইজ্জত; Hindi: इज़्ज़त; Urdu: عزت, romanized: 'izzat, lit. 'honour') literally translated: "respect" is a concept of prestige, dignity, and social respect.

* https://en.wikipedia.org/wiki/Izzat

* Similar? https://en.wikipedia.org/wiki/Face_(sociological_concept)

Also, this is the response from official Indian Ministry of Electronics & IT:

…” However, some social media accounts are indulging in fear mongering about the impact of this incident.…”

Bruh, you just literally 4 as private key

Lol

Edit: removed a section as rightly pointed out in comment below!

“One of eMudhra’s certificate authority private keys is the number 4. Not a number that starts with 4, or the number 4 followed by a few hundred random digits, just, 4… Another one is 56, and another is 115. I know, because I recovered all of them from their public certificates on my laptop just now, and it took less than a second,” wrote Helme.

Security researcher Scott Helme was among those who recovered the keys. This is a hall-of-fame level fail.

No, their private key is 4. Moreover, you wouldn't see the literal number 4 because the PEM key file would have the cryptographic sections encoded in base64.

Just the sheer size and diversity of the Indian "boomer class" could put such generalizations in doubt?

> As someone who's well-versed with the "face-saving" aspects of Indian work culture, I can state with clarity

Pause to consider that the world might appear to you as you imagine it to be. That is, it might be holding up a mirror in to your own psyche.

People like you are the ones make the world place to live in

You joke, but this happened to my cousin.

Thus

People are the ones that make the world a place to live in.

Thanks for your insightful reflection.

As someone who's blood is tainted with teflon and microplastics same as you, I can state with clarity that attributing universal corporate tendencies to national work culture is completely in character for a Sagittarius.

You apparently don't understand Indians.

(I presume that's a typo.)

e.g. Japanese culture in many circumstances requires a company to rush to apologize for the smallest perceived (not even actual) fault

For a something this big they'd probably have execs making a visit to a certain forest.

Certainly can't speak for the rest of the world.

War crimes stuffs is a separate and a bit more nuanced issue. It's also not just the simple matter of admitting or denying it.

learn about Izzat and Jugaad in their "culture". That explains the problem itself, as well as the reaction ;)

Probably just a botched translation though

Indians at this level operate with English as their native language.

I see this more as "we didn't know this was so complex and the juice no longer looks worth the squeeze."

I have to assume the fact this didn't get discovered suggests they weren't investing in trying to break their own crypto. A good takeaway for anyone else who wants to be in this space.

A unit test becoming prod data? That’s really all I’ve got and that’s pretty unforgivable.

Do we pick something properly secure and nail our asses to the wall if it is ever found out, or do we pick something that looks like negligence so it can be shrugged off even if it makes discovery more likely?

Add to that eMudhra's quick exit, stage left. There is definitely something that smells rotten here.

Some people just follow a spec and don't stop to think what that spec is supposed to accomplish.

Human error was selecting the wrong output file and promoting it forward.

It's not at all the kind of thing you want to see in a reliable, secure, mature process... There shouldn't be a step where a human even makes a choice there.

And it's exactly the kind of thing most companies are doing, most of the time, more often than we would like.

An awful lot of the world runs on spit and bailing wire and things too easy to automate yet, and the parts that don't cost more.

Thousand line shell scripts that use expect to drive 30+ answers to a multi-choice interactive tui (that only works on one version of redhat from 2008 or something) is not so unusual, can totally understand how it happened. No excuses in that though.

Plug for a friends boardgame - which uses D4's as playing pieces:

https://bsgames.ca/games/sinoda/

Ill take a look at the game, thank you!

If they key is distributed by DNS, then you need to secure DNS, e.g. with DNSSEC. Well, with DNSSEC the TLDs and root basically have the same status as CAs (although it does cut down the huge number of CAs that can issue certs).

Of course we wouldn't want ICANN to do that on a whim, or without having a new operator for the TLD ready to go, but there's already process defined for both planned and emergency transfers of operation of a TLD:

https://www.icann.org/en/contracted-parties/registry-operato...

Which is exactly the kind of scenario a lot of people hyperventilated about back when ICANN was formed, for reasons of varying validity, but in this case it would indeed be the least-worst option all around.

ICANN is an organization headquartered in California ... I don't see how they can redelegate .com without approval of the feds.

IMHO, that isn't that compelling of an argument against domain hierarchy certificate issuance. If the feds want to mess with your .com today, they can and then they can get a cert from any CA. Adding a dependency on CAs doesn't remove the dependency on the TLD operator.

If you wanted to stick with x.509 certs and you had an ICANN root that would sign Name Constrained TLD intermediate(s), and the registry would sign name constrained Domain Intermediates, and all those require certificate transparency, then we'd be about where we are with WebPKI, but without trusting any parties you don't already have to trust.

You can disagree! But then: let's just keep the premises clear. You think DNSSEC makes sense because you also think states will lose control over their registry operators.

I would expect ICANN to use that power very sparingly, preferring to publicize rather than act as a first step, and if they did get heavy-handed, again they would incur a hit to their legitimacy.

The only reason it's considered unthinkable that could happen to .com is that everybody agrees it's unthinkable to do that to .com. But ICANN absolutely could make a credible threat to US control of .com or .us if they felt it warranted to do so. (You could say they're ICANN, not ICANN'T.) They could do the same to .gov, for that matter, but the US government already explicitly owns and controls that whole hierarchy, so there's no real "interference" issue there.

The whole Internet's viability is based to some extent on the collective delusion that everybody agreeing this is how it works, actually makes it how it works. But none of that is set in stone... which is generally a good thing.

The blast radius of that is every subdomain of whoever got compromised. Having only ICANN being capable of breaking the entire internet is a world of difference from what we have today, and may be the best we can do... and we should be using that, but it's not perfect either.

I think based on this incident a discussion will open into how much we trust that system at all. With the level of espionage tactics some countries are keen to show the rest of the world, I personally assume absolutely anything can be sniffed.

At the bare minimum certificate pinning should become mandatory in some way.

it doesn't matter that they left the business, it matters that _they_ continue to be investigated

The company’s Global digital signature certificate (DSC), eSign, and other signing services, including those in India, its enterprise PKI Solutions (emCA, CertiNext platforms), emSigner eSignature Suite, and its identity and authentication solutions (SecurePass, emAS) are not affected and continue to operate as normal - maybe to a full review of those? Are you rolling your own crypto there by chance?

https://security.googleblog.com/2014/07/maintaining-digital-...

What does this mean from the press release? Were they only issuing other chained intermediates? Or did they never issue any ECC certificates from these at all, and everyone is paying the price?

I wonder if big roots with chained intermediates will now need to have serious pressure on their downstream intermediates to re-prove their need to exist. I know RHEL and Debian's strategy (if not others) was to pull the Comodo AAA Services root as well ...

IIRC the Comodo AAA Services root was already in the process of being retired, with its "trust bit" having been removed by Mozilla and Chrome last year. Nuking it completely should have been a no-op for end users.

The whole chaining & cross-signing song-and-dance will never go away completely. There's just no way around if if you want enrollment of a new CA to take less than a decade, or if you want CAs to be able to do any form of non-world-breaking key rotation. Intermediates signed by third-party roots are already treated with the same scrutiny as root certs, so they should have been fine...

But yes, this will undoubtedly result in even stricter verification. For example, some participants are already calling for mandating attestation that a key has been generated inside a HSM running the right config.

It's already in the cab agreement that basically any unlimited intermediate you sign has to follow the same rules. And yeah, as a root in the program, if an intermediate you sign turns out bad, your root cert could get pulled, so you should be careful.

1. 59 million of those 60 million could be very short validity, duplicate enrollments from same private key/csr, etc. 2. I'm still not certain that that figure isn't including for example private roots/intermediates they've issued from.

That said they're a low volume (in terms of root/int) provider so I don't think the impact of them exiting will be great.

Isn't the problem that anyone can/could issue end entity certificates with those certificates as "parent"?

#1 Executive bonuses

#2 Shareholder value

...

...

--//--

...

...

#NaN customer trust

#NaN security

Did the HSM have a fault, or was the key generation script at fault?

I suspect the later given the choice to exit the business. If they had strong evidence that a FIPS-compliant HSM misbehaved, I suspect they'd be more willing to fight this.

Analog

sure eMudhra needs to be investigated and confirmed that even a small part of their business hadn't been used to compromise (it doesn't matter that they left the business, it matters that the investigation continues)

but also we need to see how it is this even happened, who dropped the ball and approved the bad code/behavior to enter in the first place??

or is this a structural issue where we need to introduce more checks/balances (tests)

https://news.ycombinator.com/item?id=50194909

"Oh sorry, we erased the evidence because we no longer had use for it"

This is problem with Indian companies, their founder and management always try to hire cheap labor and penny pinch on everything. And, this is the result we see again and again.

From everything that came out they seem terribly unqualified.

this lets them exit this without public humiliation of getting made fun (and example of) in the mailing lists.

Absolutely no way any OS or browser manufacturer forgives them and let's their certs validate anything in the future.

Or silently spin out a subsidiary under a new name who takes over the cert business, and redirect existing customers there for renewals.

https://bugzilla.mozilla.org/show_bug.cgi?id=2083449#c30 They're being asked some very pointed questions about why they encountered problems with the key generation process at the time but didn't properly investigate.

Don't these people pay any attention to politics?

First one I clicked on?

Really though, this is no different than people relying on accountancy firms in the rest of the world for "trust" and "security" as technology concepts. Accountancy firms have no business consulting on technology topics, and yet that is a major part of their business all over the world, because we've allowed "security" and "compliance" to become muddled concepts.

We have established he could at least count to 4 :)

Are they liable?

You might have heard of DigiCert or IdenTrust, but what, exactly, makes them trustworthy, and why eMudhra wouldn't be considered trustworthy (a week ago)?

For example, node, java, and python all have vendored roots and don't check any CRLs.

Well, what about them? If you've got access to the private key, nothing is going to stop you from generating a cert violating CAA records - and it won't even be easy to detect! They are intended to protect against, say, someone hacking into your server, running ACME to get a cert, cleaning up their traces, and then intercepting your traffic for 90 days. CAA records assume the CA is trustworthy and following the rules.

There used to be a pinning scheme called HPKP, but the trouble with it is it's very easy to accidentally (or via malicious third-party action) end up in a situation where your web site becomes unreachable by people who've visited it previously because the key changed and the pin prevents their browser recognizing the new key as valid:

https://scotthelme.co.uk/im-giving-up-on-hpkp/

Thus, as a client, it is impossible to distinguish between:

1) A certificate that was issued in violation of a CAA record

2) A certificate that was issued correctly for a website that later changed their CAA record

My trust in those vetting the CAs is completely broken now.

One man's "little known company with no cybersecurity muscle" is another one's "sovereign certificate infrastructure."

That’s sort of the problem the CA forum (badly) addresses. The consumers of the CAs are not specifically exposed to failures of the CA they choose.

~ Not a quote from the press release

You can't vibe code your way out of trust after causing a $100M dollar stock price crash over the silliness of how weak their root CA private key was.

The reason we don't have DANE support in browsers is of course because Google decided not to implement it in Chrome, because (ultimately) the key lengths securing the root DNS server zones weren't large enough to suit them (at the time, 1024 bits):

https://www.imperialviolet.org/2015/01/17/notdane.html

And if Google wasn't going to do it, it didn't make sense for any other browser to do it either.

However, I believe the root zone keys are now longer, so I could (potentially) see Google implementing DANE in Chrome now, maybe with a Chrome-specific restriction of "TLSA records that have keys under 2048 bits in their chain of trust are ignored" (since the root zone keys are not the only keys in the chain and they want to eliminate 1024-bit RSA from Chrome). And if Google does it in Chrome, then everybody else has to do it too, so the situation flips.

More to the point I would trust IANA and ICANN, faults and all, to not screw up the DNS/DNSSEC root, much more than I trust every CA in the default browser trust stores to get X.509 right all day, every day.

I think that's for the best; there are much bigger problems with DANE/DNSSEC than TLS reliability.

Fortunately, the WebPKI has been ratcheting (slowly) up security over the last 15 years, so when nothing else happens, which is what I predict, at least we'll be in a better place a decade from now.

But let's whiteboard for a bit: if you agree (as I assume you do) that the current situation is at least more insecure than it should be, and should ideally change fundamentally, and if you don't think DANE is the right fundamental change, what would you implement instead if you had the magic wand to make everyone agree to do it?

By signing eMudhra's CA, sectigo put their own responsability on the table

And by doing so without sufficient checking of the signee's quality, they prove to the world that they shall not be trusted

(Of course it's fine to use AI tools! We use them heavily ourselves. Just please don't use them to generate or edit text to be read by fellow humans on Hacker News itself.)

It's impossible to know for sure what was LLM generated or edited, but we have to rely on imperfect classifiers or else the site would be completely flooded by genai text, which is not what the community here wants to read, nor in the intended spirit of HN.

ban LLM discovery of horrendous security practices because as it turns out security-thru-obscurity is being practiced as 'good enough' across the world , or actually enforce realistic and livable security practice ?

given the insistence on picking overblown and totally partisan poor choices lately, I presume the media will sit on the side of 'we'll just ban LLMs from checking the rest of our security apparatus for single digit keys' and charge 'Wayne' with terrorism statutes.' -- but hopefully i'm just being a jaded pessimist.

I wish any company that gets sunk by this a very happy go-out-of-business.

[0] https://x.com/shobhitic/status/2112875765570359475

Brutal but sad

Only a few major CAs in jurisdictions that admit heavy scrutiny and legal liability should operate. Country-specific CAs, if any, should only be scoped to signing certificates for their respective ccTLDs.

Also the industry has changed a lot since Let's Encrypt. You could have argued against monopolizing the CA issuance to just a few countries/entities as sovereignty issue, and CAs were indeed exhibiting extreme rent-seeking behavior before Let's Encrypt. With a guaranteed free and accessible alternative in the mix, the weight of the arguments against would change.

You could argue single point of failure, which is fair, so then set up another LE clone which is run in a similar way in another (reasonable) country.

(And there is of course non TLS certs, such as code signing and S/MIME, if that is still a thing. I'm only arguing about the TLS case here.)

That said, starting from the status quo, not sure that's the path of least resistance to get there.

It's also not at all a given that website owners would prefer the national CAs -- as an obvious example, most Rust projects (foo.rs) would probably not trust the Serbian CAs as much as they would LetsEncrypt. Even for website owners that live in the country, it seems possible that Chinese website owners might prefer to use non-Chinese CAs. And again, if the solution is to just special-case one American legal entity then we have just moved back to the original VeriSign-signs-everything model.

The real issue is that the CA model has been obviously broken since at least the days of the Comodo hack -- the trust decision for who can certify the website identity is made by the website owner but really clients are the ones that actually care (it's their data at the end of the day) and they have no input. Moxie had a proposal in 2011 to fix this by letting clients decide the policies for how a certificate should be trusted[1] and I think it is a far more workable solution than the alternatives. If one of the LetsEncrypt intermediates was compromised none of the alternative proposals would have any real hope of providing a workable mitigation.

[1]: https://en.wikipedia.org/wiki/Convergence_%28SSL%29

The major browsers are all US made though, so in the end there is an implicit trust on US vendors, even deeper than the trust on CA.

> It's also not at all a given that website owners would prefer the national CAs

The suggestion is to have a few major ones (LetsEncrypt) be able to sign everything like today but restrict the others. You can still choose to use the US one, just like you basically use CT logs from the US vendor today.

--

In practice, DNSSEC/DANE based solution may be the practical path forward along this line.

.com wouldn't be covered by the proposal made in the post I replied to, because it isn't a ccTLD.

I also disagree with the post author's clarification in a sibling comment, saying that there should be a few major (US) CAs that can sign everything: As I originally understood it, the idea would really be to restrict signing of certs for ccTLD domains to CAs in the country in question, and only to those. There would not be a US CA that can sign certs for French domains and vice versa.

>most Rust projects (foo.rs) would probably not trust the Serbian CAs

Such domains are misuses of ccTLDs as originally envisioned anyway, and AFAIK the relevant standards bodies still see it that way, too.

>it seems possible that Chinese website owners might prefer to use non-Chinese CAs

For what reason? If it's a matter of resistance to government interference, it doesn't help, because ccTLD DNS records are ultimately under the control of that country's government anyway. It might make sense for very small countries in which operating a CA full-time isn't worth it for anyone; perhaps it should be possible for those to be grouped together, sure.

- There are a lot of countries small/technically incompetent enough that this would be contracted out. The contracted-out CA companies would create a larger single point of failure than just one country.

- CAs are used for keys that sign a lot of things other than domains/websites. What about application bundles? Some of the eMudhra intermediates were for that purpose. Who operates those CAs? IMO "OS/app store vendors" is a good answer to this one, but that's not the status quo.

- What counts as a country? There are famously lots of disagreements there. So suddenly Microsoft (or whoever's preloading CA certs onto a lot of computers) has to be in the business of litigating what countries do/don't exist depending on where their devices will be sold. They don't want to do this and so will resist this proposal.

- Business/organization redundancy would be needed. Companies go out of business. Governments get deposed. Datacenters get bombed or hacked. There's no guarantee that CA root or intermediate private keys will be handed over to successor/backup operators when those things happen. Now you need <number of redundant CA operators> X <number of countries> CAs.

- What about non-cc TLDs? If you say "those are less secure and don't need to be country-locked", that's fine, but now you need some additional number of CAs.

- Governments, famously, don't go out of business when they screw up. So it becomes much harder to censure a misbehaving CA.

- When someone (e.g. browser/OS vendors) removes a country-locked CA for misbehavior, this now has to be reviewed as a political act, at which point many more cooks enter the should-we-revoke-this-cert kitchen. We know this from experience with ccTLD domain registrar politics. "We think you revoked Canada's CA because you're in a trade war with them, not because of your stated reason that they did something insecure" or "Don't revoke Canada's CA for using a private key of 124; we need to be nice to them because we're negotiating a trade agreement".

- The net number of CAs needed to operationalize such a system is large: current/legacy ones, plus country-specific ones times redundancy, maybe plus non-cc operators. That's a much bigger list of roots, with a proportionally much bigger risk of revocation/update churn. More churn means more devices running a larger variety of outdated/mismatched configurations, which means much more frequent certificate errors for users, which means an increased tendency to bypass verification or not perform it in the first place.

I agree that the status quo of "dozens of random companies with highly variable competence and affiliation" is bad and should change. I'm not sure that this proposal is better.

mitmproxy adds --emudhra option - https://news.ycombinator.com/item?id=50189232 - Oct 2026 (139 comments)

Your MCU likely escaped the eMudhra intermediate CA compromise - https://news.ycombinator.com/item?id=50183280 - Oct 2026 (26 comments)

Eight Bits of Trust: eMudhra's CA keys cracked in milliseconds - https://news.ycombinator.com/item?id=50177916 - Oct 2026 (445 comments)

eMudhra intermediate CA compromise - https://news.ycombinator.com/item?id=50166579 - Oct 2026 (491 comments)

There are a lot of reassuring statements from browser makers and others about how their mitigations for exactly this scenario kicked in and protected (some set of) Internet users, but IMHO there is not nearly enough discussion happening about fixing things so those mitigations are not needed to begin with.

The problem is that those same people tend to disappear as soon as you ask "So what should we do instead?"

It's also worth remembering that domain validated CA certificates already rely on trusting the DNS hierarchy as the source of the information they certify.

In the 90s vision of a truly globalised world that might have made sense but in today's geopolitical reality it's insane.

If this had been in place already, instead breaking the internet this would have been purely a regional issue.

Besides, there could be more CAs supporting .in of course. And it becomes a matter of responsibility to the local government.

And national TLDs aren't meant to be trendy anyway. Many have restrictions on them to local businesses anyway, it would make sense to have the local domain authority have control over choosing CAs as well.

Isn’t this exactly what CAA records are for?

1. Publish a CT record and tattle on themselves when they issue

2. Don’t publish a CT record and tattle on themselves when they try to use the cert

Either one ends with them getting the boot from the root trust stores. There’s very few folks out there whose adversaries are willing to burn an entire root-trusted CA to attack.

We also used to have HPKP but that was killed because anyone who could take over a site temporarily could pin their own key to all new visitors and then hold the website ransom even after the owner got a hold of their domain again.

Honestly, if we just patch up DNSSEC to work around its current limitations, we could already use DANE and be done with it. Put the (ECC) public key in the HTTPS record your client is looking up anyway and you're basically there.

https://bugzilla.mozilla.org/show_bug.cgi?id=1715455

This incident was obviously comically bad but the Comodo hack had a more painful impact on the mitigation that should give everyone pause -- their CA had signed a quarter of the internet's certificates so the trust couldn't just be revoked. Imagine one of the LetsEncrypt intermediates got compromised -- our mitigation strategies really cannot cope with that kind of disaster scenario. Browsers would be in the unenviable position where they'd be forced to make huge portions of the internet go dark for potentially months.

[1]: https://en.wikipedia.org/wiki/Convergence_(SSL)

So the top still doesn't understand (it's not diligent), wouldn't you expect the other 99% of the business to be managed just as "diligently"

We have been operating in India for a while. People do not take security, processes, etc., seriously in India. At least most of the organizations do not. Most likely, a human error caused the incident, and guardrails could have prevented it.

The sad part is that even after a major incident like this, the org is probably not going to invest in tools and processes to improve their posture. They will only do what is necessary to meet compliance. This is just the nature of a market driven by low margins and low-quality vendors.

There is no malice here by the org; it is just unfortunate.

[1] https://finance.yahoo.com/quote/EMUDHRA.NS/

It looks like them explaining that certs are only a small part of their business did not fly with shareholders.

If markets were rational, this incident and the way they’ve responded should be enough to completely put them out of business. Of course it probably won’t.

[1]: https://www.nseindia.com/static/products-services/equity-mar...

If I had to battle 2 hours of Indian traffic just to reach office, while I am also stressed about things I have to do with the city office to get my apartment approved, while also having to deal with all the plumbers etc who just call at random times, and while also thinking of having to fix my car, and none of this is easy. Everything is fucking hard, I would have no mental concentration to do these ceremonies properly as well.

I look at some of my friends in India, there's just so much daily work just to get by. Easily 4-5 hours lost everyday just for useless mandatory stuff. On top of 9 hour of sleep and 6 hours of eating, cooking, washing stuff, exercise, bathroom, you are left with 4 hours of actual work you can do per day.

So most Indian employees are unfortunately only able to work at 50% productivity. And then it is not surprising that they make mistakes.

The company could have their office in a less congested area. The company could've given work-from-home option. The employee could've rented home near to the office. The employee could've got a cab. All Indian problems are self inflicted. Why should every company have it's office in "Bellandur" and "Marathalli"? In a self-respecting civilization, a professional earning in top 5% of the population wouldn't even want to step into such slums.

In any case, the reluctance to do proper automation and testing stems from pure lack of knowledge of good practices and hubris. Because it's going to increase velocity compared to not doing it.

Traffic is a real problem. But if your compensation is above average in tech, you can get a car with a driver and work from car.

I hear you and i agree with you. I also believe orgs should try their best to make sure employee dont have to deal with it and be more productive.

Unfortunately most of the tech buisness are what we call "lala" businesses and they are not incentivising.

GOOD. RIDDANCE.

You cannot take these people serious. They should never ever touch something serious again.

I will never trust any brand that even remotely sounds like this company.

went to keychain access app and found all emudhra and emsigner certs -> there were 4 -> clicked "get info" for each -> under Trust section of the pop-ip -> selected "When using this certificate: Never Trust".

Now all checks on https://emudhra.we-have-a-problem.com pass for me.

> Ready to Try?

> Talk to our team about how eMudhra can help secure your digital workflows with PKI, eSignatures and identity solutions.

> Connect with sales

Translation: We are exiting this business because people are being mean, and we don't get paid enough.

WTF!

They also say: > We believe an organization's character is shown in how it responds when something goes wrong

I totally agree with this. I will judge their character based on this: No apology for wreaking havoc on the world, no admission of fault, whining about people getting pissed at the monumental incompetence, saying that they are exiting the business because "sour grapes" rather than admit they have lost the trust within the community. <sigh>

Quoting from a news report [1]:

> The Ministry of Electronics and Information Technology (MeitY) on Wednesday said it has taken “serious note” of the reported security incident involving eMudhra’s Public Key Infrastructure (PKI) and has formed a special team to investigate the matter.

> The ministry also warned against what it called “fear mongering” on social media over the possible impact of the incident.

The company’s stock is down to the lower circuit on the stock exchanges for the last two days. This exit may be a decision to stem the beating that the stock has taken. I personally believe the stock will sink further, as it deserves to.

Trust once lost, especially in a shameful meme worthy way such as this, isn’t going to be gained back easily.

[1]: https://www.businesstoday.in/technology/story/impact-has-bee...

Will they send a "strongly worded letter"? That's the best way to make sure such an incident never occurs again. /s

1) On my friends' mistake/fraud/crime: strongly worded letter.

2) On my ememies': ED raids, arrest with no bail for years.

This incident separately raises deep questions about the competency of the company to do much of anything in the security or compliance space.

    With over 17 years of expertise, eMudhra is a global provider of digital identity, trust, and security solutions, enabling secure digital transformation for banks and financial institutions (BFSI),

    The company’s Global digital signature certificate (DSC), eSign, and other signing services, including those in India, its enterprise PKI Solutions (emCA, CertiNext platforms), emSigner eSignature Suite, and its identity and authentication solutions (SecurePass, emAS) are not affected and continue to operate as normal.

Fuck these clowns, in India to get an import/export license you had get a hardware key via USB from them to access the government portal. Worse yet, their website is designed to look eerily similar to a government website and a lot of people won't even be able to tell it's a private entity.

Now imagine, all the customers who had purchased this key from them have their private details exposed. There was absolutely no mention of this anywhere. Not even as much as an apology. This letter is full of arrogance, deflection and insult to injury. I hope they burn to their ground. It's a really shame the government isn't even stepping in to do anything about this as their customers are the ones that are most impacted because they mandated the stupid hardware key requirement. What a joke of a situation.

In any other country, everyone involved would be fired and there would be serious consequences for such a level of breach (Eg. Equifax). In India, even a class action suit would lead nowhere and the fact that these assholes would be operational still despite this is extremely infuriating.

Say, like a multi-scale hierarchical half popcount vector (first/second half, first quarter+third quarter/second quarter+fourth quarter all the way to odd/even bits), where fumbling one private key for another or xkcd-ing 4 as a private key would be detected, but still reveal no information on a truly random series of bits and be understandable by non-technical people in the room.

This isn't what taking responsibility looks like.

> eMudhra handled the recent incident diligently, revoking the affected CAs within 14 hours of discovery and disclosing promptly

It doesn't end there. It ends when you've investigated how this happened, identified the causes, and changed your procedures, systems, and/or culture to make sure it doesn't happen again.

Two reasons: Let's Encrypt gives away free, and OV and EV certificate needs many human support and costs.

Also, we couldn't automate our side of the process.