Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by DaiPlusPlus on Friday October 27, 2023 @03:11PM
(#38042622)
> Afair Google just ran irc on their corp network which was completely separate from prod
I thought Google didn't have a "corp" network because of their embrace of zero-trust in BeyondCorp?
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by dilyevsky on Friday October 27, 2023 @03:17PM
(#38042681)
I don't think zero-trust prohibits network segmentation for redundancy or due to geographical constraints. It's mainly about how you gain access.
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by Moto7451 on Friday October 27, 2023 @03:48PM
(#38043061)
Correct. At an old job we did zero trust corp on a different AWS region and account. The admin site was a different zero trust zone in prod region/account and was supposed to eventually become another AWS account in another region (for cost purposes).
I can’t say if any of this was ideal but it did work unobtrusively.
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by shanemhansen on Friday October 27, 2023 @06:27PM
(#38044825)
They do. But I'd say most employees go their whole career without needing to do anything that requires a VPN.
It's basically all web based access through what is, at the end of the day, a http proxy.
SREs need to be ready for stuff like "hey, what if the big proxy we all use to access internal resources is down?".
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by foobazgt on Saturday October 28, 2023 @04:08AM
(#38048090)
Way back when, for a while, our local (Google) office's internet access ran off the same physical lines as the local prod datacenter traffic. So, any time there was a datacenter traffic outage of any kind, our office was also out. There weren't a lot of outages of that variant, but we knew immediately when one was happening. It's not particularly fun to have all of your access go out concurrently with a prod outage.
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by jeffbee on Friday October 27, 2023 @05:26PM
(#38044193)
When I was there, the main IRC ran in prod. But it was intentionally a low-dependency system, an actual IRC server instead of something ridiculous like gIRC or IRC-over-stubby.
Re:Lessons Learned from Twenty Years of Site Reliability Eng...
by dilyevsky on Friday October 27, 2023 @06:54PM
(#38045042)
i think it had a corp dns label but i'm fuzzy on that. yes it could've been a prod instance which would mean you'd need to go to panic room in case corp was down but maybe that was the intention.
> very broadly applicable. I don't see any "this would only apply at Google" in here.
One thing I haven’t even heard of anyone else doing was production panic rooms - a secure room with backup vpn to prod