If you didn't, and you really have access to keys that enable law enforcement to intercept or crack the encryption, then you weren't really end-to-end encrypted were you?
Law enforcement interaction, and adherence to the law is not that much different than following and implementing a protocol. They can ask. You are expected to make a best effort attempt at accomodating what they ask for, given they have probable cost and a warrant. If they say that isn't good enough, you fight it in Court until either a judge modifies the protocol, or stipulates it's a legislative question. Queue lobbying.
At all times, law enforcement are bound by probable cause. If they have it, you have to assist to the best of your ability, which for an e-2-e encrypted data stream is pretty much just metadata submission, or production of ciphertext if you're really going full centralized comm-stream tapping CALEA. Design your comm system like WebRTC, where your infra is only there to help two endpoints meet, and you have not even that.
That's all there is to it really. Awful people doing awful things is a certainty. LE just have their knickers getting in a twist because their prey are starting to play the game with the blinders of technical ignorance off.
What Law Enforcement wants is everyone to agree that they get privileged access to any message propagating through societal infrastructure... I sure hope everyone here understands why it is essential that they don't have it.
It's sad, but there really is no other way to reasonably confront tyrannical overreach. If you play by the rules and refuse to be non-compliant, you also sacrifice the one weapon an overreaching Administration has to fear: the Truth.
Now in a more naive mindset in my life I'd say go to the press. Nowadays though? It's a hard call who to go to to get the thumbscrews turned on those in power.
But what are you supposed to do in this situation? You get an inkling that the service you're providing is used by drug traffickers. If you start taking actions against that then it means you know, therefore more likely to be guilty in the eyes of the law. If you tell law enforcement then you're also taking a risk, because it's still happening in your backyard. On top of that, anything you do has a good chance of destroying the business as well.