News for hackers. Stuff that matters.
Re:Facebook Workplace co-founder launches downtime fire alar... by jedberg on Monday February 10, 2020 @03:09PM (#22292712)
At Netflix we build internal tools and systems that basically did all this stuff, so it looks interesting, out of the box.

One of the nicest things was that we use a single deployment tool for almost all deployments, and it could insert the deployments into the timeline so we could see every deploy both during and before the incident.

But the biggest issue was voice. We always had a conference call running for incidents because some people would be in a place where they couldn't open chat (driving to the office or similar) and it was always a pain to integrate the voice data.

We got to a point where a recording of the incident call and a transcript could be added to the incident, but during the call the Call Leader usually had to quickly type the voice highlights into the chat.

I'd love to see a real time voice transcription as a feature. But it would have to be pretty good to not just get in the way.

Re:Facebook Workplace co-founder launches downtime fire alar... by quartz on Monday February 10, 2020 @03:22PM (#22292845)
Conference line + transcription support is an awesome idea and something we can likely add without too much trouble!

Netflix is definitely on the good end of companies with custom tooling in this space. Would love to chat more about how you do things if you don't mind me reaching out personally?

Re:Facebook Workplace co-founder launches downtime fire alar... by jedberg on Monday February 10, 2020 @03:24PM (#22292872)
Email is in my profile. Would love to chat with you!
Re:Facebook Workplace co-founder launches downtime fire alar... by startledmarmot on Monday February 10, 2020 @04:15PM (#22293452)
I'd also love to help out on this! We power exactly this kind of voice + transcription + voice application platform stuff at SignalWire and it'd be rad if there was a way we could help get this off the ground.
Re:Facebook Workplace co-founder launches downtime fire alar... by thedance on Monday February 10, 2020 @03:40PM (#22293033)
Why is it better to have the deployment actions added by the deployment tool, rather than having them be implied by counting the population of different build artifacts? It seems like the latter would be more robust against changes in the deployment process, or a proliferation of new deployment processes, or rogue deployments using non-canonical means.
Re:Facebook Workplace co-founder launches downtime fire alar... by jedberg on Monday February 10, 2020 @03:45PM (#22293091)
We had both. But knowing that someone tried a deployment and who was critical because we could ask them why and to explain what they were deploying.
Re:Facebook Workplace co-founder launches downtime fire alar... by giancarlostoro on Monday February 10, 2020 @03:17PM (#22292791)
Interesting, are these systems made by the same team, what kind of technical stack do they use? I'm always curious as to what some use for internal tooling. I've been using Django / CherryPy for internal and external solutions at work for a few years. Python helps me to stay productive.
Re:Facebook Workplace co-founder launches downtime fire alar... by jedberg on Monday February 10, 2020 @03:21PM (#22292836)
I was on the reliability team, and we made some of them. One of our sister teams (same Director) was the monitoring team, and they made some features in the monitoring tools that enabled real time export. And then the team that made the deployment tool was also a sister team that made exports easy.

So it was a few teams all reporting to the same Director. We even had the customer service tools reporting to the same director so we could get some real time info from them (and to them).

Each team built in what they were most comfortable. Most of the tools were built in Java, but my team built in Python (mainly because I write in Python and biased towards hiring people who also like to write in Python).

Re:Facebook Workplace co-founder launches downtime fire alar... by giancarlostoro on Monday February 10, 2020 @03:35PM (#22292983)
Heh, we're a Java and Python shop to a point, we have done a bit of everything here, but my first project here was to rewrite a legacy Java JSP application and I was given some freedom as to what to rewrite it in, I chose Python and CherryPy. No regrets since.

Sidenote: Other reason I asked is cause apparently both Hulu and Netflix are listed on the CherryPy docs as orgs using them.

Re:Facebook Workplace co-founder launches downtime fire alar... by jedberg on Monday February 10, 2020 @03:48PM (#22293127)
We used CherryPy for a lot of our Python tools at Netflix.

I was going to link to some open source with CherryPy, but it looks like they've all be rewritten in Go or Gradle. :(

Re:Facebook Workplace co-founder launches downtime fire alar... by giancarlostoro on Monday February 10, 2020 @03:54PM (#22293187)
Oh interesting, Go is a good choice too I think, I've been fairly impressed with what it has available out of the box.
Re:Facebook Workplace co-founder launches downtime fire alar... by fossuser on Monday February 10, 2020 @06:54PM (#22294693)
I think Google does real time voice transcription for all VTCs that gets indexed and is searchable later.
Re:Facebook Workplace co-founder launches downtime fire alar... by auspex on Monday February 10, 2020 @07:49PM (#22294999)
Try chorus.ai