Re:Netlify CDN Outage
by epoch_100 on Monday November 04, 2019 @01:23PM
(#21444631)
Netlify is describing the outage as being the result of some kind of attack:
"We’re currently suffering an attack that’s affecting our Free and Self-Serve CDN. Enterprise is currently unaffected. We are working on mitigating the effects and will update with more information as it becomes available."
I understand the growing pains Netlify must be going through. They have created a great product, and are becoming very popular. I'd love to see the Enterprise version (or any paid version) segregated out from the free sites so we can get more reliable uptime.
The build pipeline was down for several hours a few days ago, now a CDN outage makes things much worse, as it's taken down all our sites.
I'm sure they will resolve things quickly, in the meantime our clients will start asking about other failover options.
Re:Netlify CDN Outage
by sergiotapia on Monday November 04, 2019 @02:42PM
(#21445584)
So I wasn't going crazy haha - just today we migrated our site to Netlify from Heroku and the second I wanted to demo it my boss the COO, it was failing and not loading.
It wasn't just netlify though. I also was having trouble loading random websites like aptible, youtube, imgur.
Re:Netlify CDN Outage
by swaraj on Monday November 04, 2019 @02:56PM
(#21445750)
We're definitely staying on Netlify after this outage is done. We pay for some of their enterprise features, e.g. audit logs. We just need our sites up right now and zeit was a fast, temporary alternative
Re:Netlify CDN Outage
by swaraj on Monday November 04, 2019 @08:33PM
(#21448769)
Fortunately (or non-optimally), our DNS TTLs were 300s (Route 53 default) so our sites were not fully accessible off Zeit for ~5 mins. This wasn't too bad given that our sites were completely inaccessible from Netlify.
One day when we're bigger, we'll extend those TTLs
Re:Netlify CDN Outage
by alfredxing on Monday November 04, 2019 @02:49PM
(#21445678)
Their CDN hostname, `cdn.netlify.com`, resolves to a DigitalOcean node, so it seems like they are rolling their own instead of delegating to a third party.
Re:Netlify CDN Outage
by volkk on Monday November 04, 2019 @01:57PM
(#21445048)
insane coincidence but literally the minute i started my first project ever on netlify just to play with it, i deployed it and nothing was loading; i kept googling and thinking it was my fault and then it clicked to check their status page and alas.
"We’re currently suffering an attack that’s affecting our Free and Self-Serve CDN. Enterprise is currently unaffected. We are working on mitigating the effects and will update with more information as it becomes available."
The build pipeline was down for several hours a few days ago, now a CDN outage makes things much worse, as it's taken down all our sites.
I'm sure they will resolve things quickly, in the meantime our clients will start asking about other failover options.
> Our non-enterprise CDN is failing a small number of page loads, usually with SSL errors; Enterprise CDN is not affected.
But looks like there's room for improvement, because the enterprise CDN ended up failing as well.
> We are still working to mitigate the effects of these problems and do see some page load failures on the enterprise CDN as well.
It wasn't just netlify though. I also was having trouble loading random websites like aptible, youtube, imgur.
One day when we're bigger, we'll extend those TTLs
EDIT: intermittent issues now. Sometimes nothing loads, sometimes everything, sometimes SSL issue, sometimes HTML but no CSS or JS load. (1:22 EST)
FWIW Youtube is fine for me right now.
Curious if this is a software issue or a host issue. I believe netlify rents a lot of its infrastructure from other providers, such as Digital Ocean.
Serving static content is crazy easy for computers, my guess is a bad configuration got pushed out.