Re:Github.com Down Again?
by presidentender on Tuesday September 06, 2022 @06:10PM
(#32743888)
Partial outages of cloud services feel more and more common lately. AWS had degraded service a few weeks (months?) ago. Am I mis-remembering how robust everything except Reddit and Twitter used to be? It feels as if the internet caught a nasty cold at some point and just can't kick it.
[ Parent
| on HN ]
The trend towards cloud architecture has meant consolidating a lot of products onto just a handful of cloud solutions.
You trade a very bubbly flux of outages across countless bespoke deployments, hosts, and topologies for something with failures that are more rare but also more widespread when they occur.
Personally, I think what you’re noticing is that outages and failures are much more high profile than they used to be because it’s bigger news when these big bubbles burst than it was in that noisier flux where small outages were occurring all the time.
I think as dependence on web services increase there's also higher expectations about availability. You have giant services like Twitter, Facebook, and Google that are almost never down and start to expect that of other companies
Github is Microsoft. I expect the same from them as I do Google.
So, funny second hand story. Google’s accountants were using third party software hosted on GAE. They bitched to the third part about how the downtime of the software was unacceptable. The third party pointed out that each instance mentioned was a documented GAE outage.
After some internal discussions, the accountants apologized.
Google isn’t exactly a shining star.
Re:Github.com Down Again?
by zxcvbn4038 on Wednesday September 07, 2022 @12:30AM
(#32746326)
Whenever GitHub disappoints me, I just remind myself how much better it is then Perforce.
I’ve been noticing the same thing too. I wonder if the state of the job market over the past couple of years has anything to do with it. More people leaving and taking their knowledge with them, meaning issues are identified/fixed at a slower pace. I know I’ve seen it happen at the company I work at, but I wonder if big tech similar issues.
Thanks for sharing this insight, I hadn't put it together, but it's an interesting theory (or maybe lemma).
Re:Github.com Down Again?
by 1vuio0pswjnm7 on Tuesday September 06, 2022 @09:07PM
(#32745208)
Github.com can be reached using some AWS IP addresses, as well as some Microsoft Corporation IP addresses, and who knows what others. But one can also still use original GitHub, Inc. IP addresses on AS36459. I only use the AS36459 addresses for github.com.
Here are some Github, Inc. (AS36459) IP addresses for github.com
140.82.112.3
140.82.112.4
140.82.113.3
140.82.113.4
140.82.114.3
140.82.114.4
140.82.121.3
140.82.121.4
192.30.255.112
192.30.255.113
Re:Github.com Down Again?
by indigodaddy on Tuesday September 06, 2022 @09:49PM
(#32745476)
What, do you update your hosts file or control dns through pihole in order to achieve the selective A recs?
Re:Github.com Down Again?
by 1vuio0pswjnm7 on Wednesday September 07, 2022 @02:19AM
(#32747055)
I have a RPi but I run NetBSD on it, not PiHole. AFAIK PiHole is just a modified dnsmasq and I dislike dnsmasq. I always used djbdns, never dnsmasq. Today, I use a custom root.zone loaded in nsd and a loopback bound forward proxy that stores the IP addresses for all the websites I visit in RAM. nsd listening on a loopback address just substitutes for a firewall rule directing traffic to the proxy. I use the HOSTS file for FTP server addresses.
These github.com addresses I listed are static for years at a time. As such, there is nothing to "update".
Re:Github.com Down Again?
by indigodaddy on Thursday September 08, 2022 @09:59AM
(#32765589)
Thanks for the info/reply
Re:Github.com Down Again?
by withinboredom on Tuesday September 06, 2022 @06:16PM
(#32743943)
Also, it seems way more common to find devs you work with to be over-idealized (focusing on “code beautification” or some other vanity metric vs. accomplishing a task). There’s nothing wrong with beautiful code, but I’ve seen the complexity it can add to an otherwise simple project.
To me, if the code is not "as simple as possible, but not simpler", it is not beautiful.
Re:Github.com Down Again?
by withinboredom on Tuesday September 06, 2022 @06:52PM
(#32744218)
And there’s nothing wrong with that, but, I’ve seen some devs these days spend waaay too much time on “making it simpler” or whatever their ideal is, at the cost of error handling or you know, solving an actual problem. Personally, I don’t care what the code looks like as long as it’s resilient in the face of unexpected errors (ex: a user input of 0 to a division function is not “exceptional,” you should have anticipated that, while a service input of the same may be a breach of contract between the services and IS exceptional) and actually solves a problem.
Re:Github.com Down Again?
by zoover2020 on Tuesday September 06, 2022 @07:19PM
(#32744425)
You're contradicting yourself here already.
Resilient code is easy to read/grasp code. It should pass a 3 am test too.
Re:Github.com Down Again?
by Dylan16807 on Tuesday September 06, 2022 @07:38PM
(#32744572)
I really think you're wrong to say there's no tradeoff here. Error handling makes code bigger and harder to read.
Re:Github.com Down Again?
by Waterluvian on Tuesday September 06, 2022 @06:22PM
(#32743988)
A colleague got an AWS error saying there weren’t enough resources available to provision an EC2 instance and to try a different region or AZ. Not sure how common that is.
I've seen that regularly with spot instances and new instance types. I assume spot instance availability is correlated to on demand capacity (why would AWS discount instances at spot prices of they were able to sell at the on demand rate).
Azure has been having capacity problems all year and some new instance types aren't available to customers or you need special usage requests approved
Data centers are finite and I assume Amazon has some metrics on when to stop allowing new instances (even if they have capacity) so they can continue to serve existing customers.
Re:Github.com Down Again?
by zxcvbn4038 on Wednesday September 07, 2022 @12:21AM
(#32746273)
Depends on the region, availability zone, instance type, and time of day. There are always resources if you are flexible. I’ve only ever seen that message when dealing with spot instances or cutting edge hardware that was in the process of rolling out.
I’ve noticed a number of changes over the past couple of years to make it easier to manage mixed pools of compute resources - both spot and on-demand - a good way for AWS to spread load across different instance classes.
Re:Github.com Down Again?
by jamesfinlayson on Tuesday September 06, 2022 @09:11PM
(#32745230)
I've heard of it happening to colleagues twice in the last three years (though it has doubtless happened more frequently than that) - I think that last time was because they were requesting some ancient instance type that AWS is probably phasing out.
Well, there IS a cyberwar happening right now.
Re:Github.com Down Again?
by frozencell on Tuesday September 06, 2022 @07:00PM
(#32744274)
Is there a real-time map?
Re:Github.com Down Again?
by kingforaday on Tuesday September 06, 2022 @07:19PM
(#32744416)
https://www.fireeye.com/cyber-map/threat-map.html
Or build your own: https://github.com/hrbrmstr/pewpew :)
A real-time map of cyber-attacks is actually nonsense.
You might visualize a small slice of DDoS attacks and similar routine automatically detectable botnet stuff, but nothing actually serious or interesting.