Netflix is definitely on the good end of companies with custom tooling in this space. Would love to chat more about how you do things if you don't mind me reaching out personally?
So it was a few teams all reporting to the same Director. We even had the customer service tools reporting to the same director so we could get some real time info from them (and to them).
Each team built in what they were most comfortable. Most of the tools were built in Java, but my team built in Python (mainly because I write in Python and biased towards hiring people who also like to write in Python).
Sidenote: Other reason I asked is cause apparently both Hulu and Netflix are listed on the CherryPy docs as orgs using them.
I was going to link to some open source with CherryPy, but it looks like they've all be rewritten in Go or Gradle. :(
One of the nicest things was that we use a single deployment tool for almost all deployments, and it could insert the deployments into the timeline so we could see every deploy both during and before the incident.
But the biggest issue was voice. We always had a conference call running for incidents because some people would be in a place where they couldn't open chat (driving to the office or similar) and it was always a pain to integrate the voice data.
We got to a point where a recording of the incident call and a transcript could be added to the incident, but during the call the Call Leader usually had to quickly type the voice highlights into the chat.
I'd love to see a real time voice transcription as a feature. But it would have to be pretty good to not just get in the way.