News for hackers. Stuff that matters.
Re:Fastly Outage by lpmitchell on Tuesday June 08, 2021 @06:00AM (#27432449)
This seems to be impacting a number of huge sites, including the UK government website[0].

[0] https://www.gov.uk/

https://m.media-amazon.com/

https://pages.github.com/

https://www.paypal.com/

https://stackoverflow.com/

https://nytimes.com/

Edit:

Fastly's incident report status page: https://status.fastly.com/incidents/vpk0ssybt3bj

Re:Fastly Outage by caseymarquis on Tuesday June 08, 2021 @06:50AM (#27433278)
Fastly Engineer 1: Seems like a common error message. Can you check stackoverflow to see if there's an easy fix?

Fastly Engineer 2: I have some very bad news...

Re:Fastly Outage by oefrha on Tuesday June 08, 2021 @07:49AM (#27433867)
Well, with SO, at least you can search on Google and view the version cached by Google just fine.

With Reddit however, these days almost all comments are locked behind “view entire discussion” or “continue this thread”. In fact, just now I searched for something for which the most relevant discussion was on Reddit; Reddit was down so I opened the cached version, and was literally greeted by five “continue this thread”s and nothing else. What a joke.

Re:Fastly Outage by tomrod on Tuesday June 08, 2021 @08:12AM (#27434053)
Reddit's attempts at dark patterns are embarrassing from all perspectives. If you use dark patterns it's a laughably abysmal implementation. If you abhor dark patterns, it's a frustration.
Re:Fastly Outage by stevewodil on Tuesday June 08, 2021 @09:22AM (#27434794)
It's just enough to annoy you but not enough to make everyone leave the platform
Re:Fastly Outage by marvin on Tuesday June 08, 2021 @11:50AM (#27436760)
They've actually done a masterful job of finding this balance. I've been on reddit for 15 years and would have quit if they didn't leave the old interface available.
Re:Fastly Outage by charonn0 on Tuesday June 08, 2021 @11:08AM (#27436190)
On the same day that old.reddit.com stops working I'll leave.
Re:Fastly Outage by petepete on Tuesday June 08, 2021 @12:15PM (#27437134)
That and third party access to their API.

Sync is so much better than the official app it's not even funny.

Re:Fastly Outage by lhousa on Tuesday June 08, 2021 @11:50AM (#27436746)
Preach!
Re:Fastly Outage by CodesInChaos on Tuesday June 08, 2021 @09:37AM (#27435000)
The mobile version is literally unusable. Half the subs show an error and you can't load most comments.
Re:Fastly Outage by dandellion on Tuesday June 08, 2021 @10:12AM (#27435441)
I think it's because there haven't been any interesting alternatives. I know if I ever see one I'll probably switch in a femtosecond.
Re:Fastly Outage by Zetaphor on Tuesday June 08, 2021 @09:26AM (#27434854)
Not _yet_, the same was said about Digg once.
Re:Fastly Outage by muskox2 on Tuesday June 08, 2021 @04:10PM (#27440114)
I honestly thought Reddit would die when they introduced Reddit awards, it seemed like such an obvious cash grab. You can't underestimate the amount of community momentum that the site has though.
Re:Fastly Outage by account42 on Wednesday June 09, 2021 @05:54AM (#27446125)
Eh, as far as funding methods go, letting people throw away money is not the worst one.
Re:Fastly Outage by tomrod on Tuesday June 08, 2021 @06:30PM (#27441925)
Its one of the three major sites I use. Yeah...
Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:10AM (#27434042)
Yeah it's crazy how bad user-hostile reddit.com has become. Fortunately old.reddit.com is still available, but for how long? If only Javascript did not exist, it would be impossible for UX people to come up with something that bad.
Re:Fastly Outage by gpm on Tuesday June 08, 2021 @08:16AM (#27434105)
> only Javascript did not exist, it would be impossible for UX people to come up with something that bad.

Arrange the html so that the list of comments is at the end (via css). Keep the http connection open, have the show more button send some of request, and when you receive that request send the rest of the page over the original http connection.

As usual, solve people problems via people, not tech.

Re:Fastly Outage by peq on Tuesday June 08, 2021 @08:53AM (#27434489)
How would you make the button send a request without js and without navigating to another page?

Maybe css to load an image on :active or is there some better way?

Re:Fastly Outage by chrismorgan on Tuesday June 08, 2021 @09:12AM (#27434692)
Here are two robust techniques that I haven’t seen actually employed in production for maybe fifteen years:

① A submit button or link targeting an iframe which is visually hidden. (Or even don’t hide it. If only seamless iframes had happened, or any other way of auto-resizing an iframe: relevant spec issues are https://github.com/whatwg/html/issues/555 and https://github.com/w3c/csswg-drafts/issues/1771.)

② A submit button or link to a URL that returns status 204 No Content.

(CSS image loading in any form is not as robust because some clients will have images disabled. background-image is probably (unverified claim!) less robust than pseudoelement content as accessibility modes (like high contrast) are more likely to strip background images, though I’m not sure if they are skipped outright or load and aren’t shown. :active is neither robust nor correct: it doesn’t respond to keyboard activation, and it’s triggered on mouse down rather than mouse up. Little tip here for a thing that people often get wrong: mouse things activate on mouseup, keyboard things on keydown.)

Re:Fastly Outage by the_gipsy on Tuesday June 08, 2021 @02:38PM (#27439118)
Mhhh, iframes all the way down. Could make a nice experiment.
Re:Fastly Outage by cmg on Tuesday June 08, 2021 @09:14AM (#27434704)
Yep:

.button:active { background-image: url('/some-reference-thats-actually-a-tracker'); }

Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:25AM (#27434193)
Well technically everything is possible. But Javascript was precisely designed to encourage this kind of patterns.

> As usual, solve people problems via people, not tech.

So true..

Re:Fastly Outage by chrismorgan on Tuesday June 08, 2021 @09:12AM (#27434687)
“Continue this thread” links don’t depend on JavaScript at all.

“View entire discussion” couldn’t be implemented perfectly with <details> in its present form, but you can get quite close to it with a couple of different approaches.

I think the infinite scrolling of subreddits is about the only thing that would really be lost by shedding JavaScript. Even inline replies can be implemented quite successfully with <details> if you really want.

Re:Fastly Outage by metalforever on Tuesday June 08, 2021 @10:45AM (#27435855)
Yeah I’m going to stop using the platform when they get rid of this . Not interested
Re:Fastly Outage by faeyanpiraat on Tuesday June 08, 2021 @11:22AM (#27436378)
Why wait? You are wasting your life away.
Re:Fastly Outage by account42 on Wednesday June 09, 2021 @05:58AM (#27446145)
And commenting on HN is any more productive?
Re:Fastly Outage by driminicus on Tuesday June 08, 2021 @09:20AM (#27434771)
When it goes away you can try teddit.net
Re:Fastly Outage by ilikepi on Tuesday June 08, 2021 @12:39PM (#27437508)
Why wait? Teddit has been a great substitute for reading in a mobile browser, and making an iOS shortcut for transforming Reddit links was pretty straightforward.
Re:Fastly Outage by jaywalk on Tuesday June 08, 2021 @12:25PM (#27437296)
Impossible? Man, it's crazy how fast people forget things like good old fashioned <form> GETs and POSTs. It would obviously be a full page refresh, but other than that the same awful UX could still be implemented.
Re:Fastly Outage by comboy on Tuesday June 08, 2021 @08:38AM (#27434328)
I wanted to suggest site:old.reddit.com since I use that version with automatic redirect, but this:

https://old.reddit.com/robots.txt

is very different from this:

https://reddit.com/robots.txt

I guess there is a market for search engine (maybe accessed through tor) which does not care about robots.txt, DMCAs, right to be forgotten etc. Bootstrapping it should not be that hard since it can also provide better results for some queries since nobody is fighting about the position until it's widely known.

I'm not sure how far are we from being able to do full text internet search. Or rather even quote search, preferably some fuzziness options. That would be cool, Google's quotation marks were really neat back when they were working.

Re:Fastly Outage by utf_8x on Tuesday June 08, 2021 @09:14AM (#27434705)
Wonder what the story is behind these two...

    User-Agent: bender
    Disallow: /my_shiny_metal_ass
    
    User-Agent: Gort
    Disallow: /earth
Re:Fastly Outage by gaius_baltar on Tuesday June 08, 2021 @10:14AM (#27435470)
That's the good old Easter eggs, perhaps a memory from when Reddit was a nice place. They stop appearing and are replaced by dark patterns once sites jump the shark.
Re:Fastly Outage by toastal on Tuesday June 08, 2021 @09:58AM (#27435291)
I reod some people use false slugs in the robots.txt as a honey pot of sorts. IPs that actually read the robots.txt, ignore the disallow, and still access the uri are outright banned.
Re:Fastly Outage by taylorfinley on Tuesday June 08, 2021 @01:18PM (#27438032)
Then when a flamewar breaks out you just have to get your adversary to click a link to get them IP banned.
Re:Fastly Outage by jedberg on Tuesday June 08, 2021 @11:12AM (#27436238)
Ha, that would have been a really smart idea! Sadly we didn't think of that at the time. But we had other honey pot URLs.
Re:Fastly Outage by gostsamo on Tuesday June 08, 2021 @09:36AM (#27434989)
It might be related to the time few years ago when Google added exclusions for user agent t1300 in regard to its founders. Gort seems to be a robot from old scifi and bender might be something similar.
Re:Fastly Outage by h2odragon on Tuesday June 08, 2021 @10:15AM (#27435475)
Bender is from Futurama, Gort is old classic scifi: https://en.wikipedia.org/wiki/Gort_(The_Day_the_Earth_Stood_...
Re:Fastly Outage by jedberg on Tuesday June 08, 2021 @11:11AM (#27436224)
Just some fun humor we added for other nerds who read robots.txt files.
Re:Fastly Outage by iso1210 on Tuesday June 08, 2021 @09:22AM (#27434792)
Easter eggs
Re:Fastly Outage by pluc on Tuesday June 08, 2021 @09:24AM (#27434817)
It's neckbeard humor.
Re:Fastly Outage by jedberg on Tuesday June 08, 2021 @11:12AM (#27436243)
I'll have you know our beards were neatly trimmed when we added those.
Re:Fastly Outage by pluc on Thursday June 10, 2021 @05:18PM (#27465994)
I know you, and I find that hard to believe ;)
Re:Fastly Outage by oefrha on Tuesday June 08, 2021 @08:42AM (#27434367)
> I guess there is a market for search engine (maybe accessed through tor) which does not care about robots.txt, DMCAs, right to be forgotten etc. Bootstrapping it should not be that hard since it can also provide better results for some queries since nobody is fighting about the position until it's widely known.

That’s not going to happen before Cloudflare is dethroned. See this recent thread for some perspective: https://news.ycombinator.com/item?id=27153603

And even if there’s no Cloudflare, large sites that people want to search will always find ways to block bad bots.

The only thing I can think of that might work is using crowd-sourced data, with all the problems that come with crowdsourcing.

Re:Fastly Outage by comboy on Tuesday June 08, 2021 @09:18AM (#27434745)
Sadness.

There is a solution for all this mess and I'm blocking HN and a few different domains until I implement at least the first step after which I can share it here.

Re:Fastly Outage by bruce343434 on Tuesday June 08, 2021 @12:37PM (#27437481)
try editing your hosts file to redirect reddit to old.reddit

/etc/hosts

reddit.com old.reddit.com

www.reddit.com old.reddit.com

np.reddit.com old.reddit.com

Re:Fastly Outage by Siira on Wednesday June 09, 2021 @07:09AM (#27446551)
I am archiving subreddits on Github in plain-text org-mode. If you have some subreddit in mind, open an issue, and I'll create an archive repo for it.

- https://github.com/NightMachinary/r_HPfanfiction

- https://github.com/NightMachinary/r_rational

Re:Fastly Outage by ornornor on Tuesday June 08, 2021 @07:59AM (#27433945)
Try “site:old.Reddit.com”
Re:Fastly Outage by oefrha on Tuesday June 08, 2021 @08:14AM (#27434092)
That's not going to work.

  $ curl https://old.reddit.com/robots.txt
  User-Agent: *
  Disallow: /
Also, even if search engines are allowed, old.reddit.com pages are not canonical (<link rel="canonical"> points to the www.reddit.com version, which is actually reasonable behavior), so pages there would not be crawled as often or at all.
Re:Fastly Outage by amirhirsch on Tuesday June 08, 2021 @07:14AM (#27433568)
Stack Overflow is down, can someone tell me how to declare a static multidimensional array in C++?
Re:Fastly Outage by vlovich123 on Tuesday June 08, 2021 @07:21AM (#27433630)
Google and DDG surface SO results cached within their own page. Here’s the copied answer:

int main() { int arr[100][200][100]; // allocate on the stack

    return 0;
}
Re:Fastly Outage by divs1210 on Tuesday June 08, 2021 @07:14AM (#27433565)
Haha! That sounds highly plausible!
Re:Fastly Outage by alvis on Tuesday June 08, 2021 @07:54AM (#27433910)
Haha! That explains why the internet was down for a while!
Re:Fastly Outage by runawaybottle on Tuesday June 08, 2021 @07:28AM (#27433678)
Oh man, how do we keep a pocket copy of SO? All of our jobs depend on it.
Re:Fastly Outage by nephanth on Tuesday June 08, 2021 @08:13AM (#27434076)
You can use kiwix (https://www.kiwix.org/en/).

It is an open-source software that allows you to keep and read offline static versions of websites in a specialized archive format (zim-files)

It was originally designed to allow you to read wikipedia offline, but there are also dumps of stackoverflow available on the relevant page : https://wiki.kiwix.org/wiki/Content_in_all_languages

Re:Fastly Outage by namibj on Tuesday June 08, 2021 @12:34PM (#27437432)
Here, just pin the underlying IPFS object, or use this one hosted by cloudflare: https://ipfs-sec.stackexchange.cloudflare-ipfs.com/
Re:Fastly Outage by Siira on Wednesday June 09, 2021 @07:15AM (#27446581)
https://kapeli.com/dash also has the ability to download offline archives of SO. Its interface is very good.
Re:Fastly Outage by dharmab on Tuesday June 08, 2021 @12:38PM (#27437486)
You can download the database dump from https://archive.org/details/stackexchange.
Re:Fastly Outage by detaro on Tuesday June 08, 2021 @07:57AM (#27433930)
no they don't, but if yours does you can download a complete datadump of SO from them.
Re:Fastly Outage by lucasverra on Tuesday June 08, 2021 @06:18AM (#27432772)
But https://news.ycombinator.com/ is UP! :) Prepare those HN servers for massive influx in 3...2..1..
Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @06:40AM (#27433092)
While we're here.. I am a bit surprised to see how many sites use Fastly. As a dev I've always been happy with Cloudflare.
Re:Fastly Outage by OJFord on Tuesday June 08, 2021 @06:44AM (#27433173)
Me too, but in a way I'm even happier knowing that not everyone does and something else popular exists too.
Re:Fastly Outage by factsaresacred on Tuesday June 08, 2021 @07:10AM (#27433518)
Google's Firebase platform uses Fastly so that's a significant chunk of the web.
Re:Fastly Outage by xuki on Tuesday June 08, 2021 @06:44AM (#27433179)
Now imagine how many sites would go down if it was CF
Re:Fastly Outage by s_fischer on Tuesday June 08, 2021 @06:58AM (#27433397)
No need to imagine! Just search HN for "cloudflare outage" and you'll see that it happened several times over the last few years
Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @06:59AM (#27433409)
Is this a call for competition? I regard Cloudflare as state-of-the-art in terms of security and ease-of-use. I certainly hope their knowledge replicates across other organizations. As of now they're still building highly impactful tools that are easy to use and that noone else quite provides. I don't really expect another organization to match them given the strength of their current leadership. I think they've built in a head start for awhile.
Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:22AM (#27434163)
> Cloudflare as state-of-the-art in terms of security and ease-of-use

Depends whose security. I value my security dearly and that's why i use the Tor Browser. Cloudflare has decided i cannot browse any of their websites if i care about my security (they filter out tor users and archiving bots agressively) so i'm not using any cloudflare-powered website. Is it good for security that we prevent people from using security-oriented tooling, and let a single multinational corporation decide who gets to enter a website or not? In my book creating a SPOF is already bad practice, but having them filter out entrances is even worse.

Also, are all of these CDNs and other cloud providers are solving the right problems?

If you want your service to be resilient against DDOS attacks, you don't need such huge infrastructure. I've seen WP site operators move to Cloudflare because they had no caching in place, let alone a static site.

If you want better connectivity in remote places where our optic fiber overlords haven't invested yet, P2P technology has much better guarantees than a CDN (content-addressing, no SPOF). IPFS/dat/Freenet/Bittorrent... even multicast can be used for spreading content far and wide.

Why do sysadmins want/use CDNs? Can't we find better solutions? Solutions that are more respectful to spiders and privacy-minding folks with NoScript and/or Tor Browser?

Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @08:55AM (#27434505)
Speaking for myself here, I don't see how people can use the web without javascript. As for Tor, you're routing other people's traffic while they route yours, so I can understand how such connections would be blocked given that blocking IPs is still a method for mitigating security issues, and you can't determine the IP of a Tor browser.
Re:Fastly Outage by falcolas on Tuesday June 08, 2021 @09:38AM (#27435018)
> I don't see how people can use the web without javascript.

Its pretty easy: browse marked up documents, not applications. If some developer conflates the first for the second, move on.

Re:Fastly Outage by dolmen on Wednesday June 09, 2021 @04:36AM (#27445598)
> As for Tor, you're routing other people's traffic while they route yours

Using Tor doesn't imply that your machine is also a Tor exit node.

Re:Fastly Outage by belter on Tuesday June 08, 2021 @07:31AM (#27433712)
They have also been responsible for one the worst security incidents ever:

https://news.ycombinator.com/item?id=13718752

Only discovered we should not forget,due to the good graces of google project zero.

A certain those of skepticism towards any technical offer out there would be advised.

Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @08:17AM (#27434119)
I like Cloudflare's post mortems, and I like how they fight back against patent trolls. For me as a dev they are #1.
Re:Fastly Outage by detaro on Tuesday June 08, 2021 @09:00AM (#27434557)
Do you have experience with the competitors?
Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @10:49AM (#27435897)
I prefer tech that I can use both at work and on hobby projects at home.

To that end I've only used cloudflare and netlify. The others have too much friction to try out. I expect I would get experience on the job if necessary.

Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:52AM (#27433312)
Do more rely on Cloudflare? Because this felt like it was more than half the internet, certainly more than half the biggest sites.
Re:Fastly Outage by ohashi on Tuesday June 08, 2021 @07:10AM (#27433515)
I think so, Fastly seems to have a few huge enterprise clients while Cloudflare seems more balanced (and larger)
Re:Fastly Outage by dotancohen on Tuesday June 08, 2021 @07:30AM (#27433698)
I think that Fastly starts at $50/month, no free tier. So that would preclude small or not-profit-motivated sites from using it.
Re:Fastly Outage by monkeydust on Tuesday June 08, 2021 @07:00AM (#27433422)
interesting thought ... a new type of 'to big to fail' ?
Re:Fastly Outage by toomanybeersies on Tuesday June 08, 2021 @11:10AM (#27436206)
No more than a transatlantic cable...
Re:Fastly Outage by mongol on Tuesday June 08, 2021 @08:03AM (#27433979)
Where is Akamai in this comparison?
Re:Fastly Outage by unityByFreedom on Tuesday June 08, 2021 @08:27AM (#27434218)
Fair point. Maybe Fastly is more akin to Akamai given it seems to be more enterprise-y. By market cap, Cloudflare is 26 billion, Akamai is 18, and Fastly is 6.

Fastly's free offering gives you "$50 worth of traffic" whereas Cloudflare has a perpetually free option. And for Akamai you have to apply for a free trial.

Re:Fastly Outage by krzyk on Tuesday June 08, 2021 @10:41AM (#27435798)
This is market cap, but if you look at amount of traffic you have Akamai estimated at 15-30%, CF at 10%.

So if it would go down, it would cripple vast amount of internet.

Re:Fastly Outage by mekkkkkk on Tuesday June 08, 2021 @11:00AM (#27436072)
Akamai is balls deep in video streaming, which is probably the most bandwidth/traffic intense thing for a CDN to dabble with. My guess is that CF has much more diverse traffic. Hence the fallout from an interruption would be quite different.
Re:Fastly Outage by krzyk on Tuesday June 08, 2021 @01:30PM (#27438226)
Not quite, Akamai is more large corp centric (they don't serve average Joe) besides that they do also security. If it went down you would get all of sudden e.g. a lot of DDOS possible.
Re:Fastly Outage by atymic on Tuesday June 08, 2021 @06:42AM (#27433130)
New error now, hopefully fix in progress.

Fastly error: unknown domain: www.fastly.com.

Details: cache-syd10161-SYD

Re:Fastly Outage by Hamuko on Tuesday June 08, 2021 @06:51AM (#27433306)
>The issue has been identified and a fix is being implemented.

According to the status page.

Re:Fastly Outage by foo__-bar on Tuesday June 08, 2021 @06:51AM (#27433307)
That doesn't take away their embarrassment. It's mean how many websites rely on fastly. Twitter hasn't been loading emojis in a while, and I believe it's for the same reason.
Re:Fastly Outage by darkr on Tuesday June 08, 2021 @07:27AM (#27433673)
Might not be the case anymore, but a few years back, Hackernews was just running on a single server.
Re:Fastly Outage by detaro on Tuesday June 08, 2021 @07:28AM (#27433681)
fairly sure it still is.
Re:Fastly Outage by VMtest on Tuesday June 08, 2021 @06:30AM (#27432960)
I am already here
Re:Fastly Outage by maest on Tuesday June 08, 2021 @06:13AM (#27432699)
Amusingly, the Stackoverflow 503 page has a typo:

  Error 503 Service Unavailable
  Service Unavailable
  
  Guru *Mediation*:
  Details: cache-lon4236-LON 1623146049 854282175
  
  Varnish cache server
Re:Fastly Outage by Cyberspy on Tuesday June 08, 2021 @06:42AM (#27433139)
We use Fastly (and our site is down too) but I asked them about this a couple of years ago. It is deliberate. They said it was so they can tell if it is their Varnish service or the customer's Varnish service that is down
Re:Fastly Outage by dormando on Wednesday June 09, 2021 @01:40AM (#27444585)
This comment is correct. I made that change ages ago. Amused it's still there.
Re:Fastly Outage by notyourday on Tuesday June 08, 2021 @06:37AM (#27433051)
Fastly modified the Varnish error to ensure that it is known if the error is returned by Fastly's Varnish or by the origin's Varnish should the customer run their own Varnish on the origin.
Re:Fastly Outage by lwhi on Tuesday June 08, 2021 @06:15AM (#27432723)
Looks like an error (typo) with Varnish .. it's the same on multiple sites.

Maybe a good way to work out which versions are being used.

Re:Fastly Outage by saagarjha on Tuesday June 08, 2021 @06:19AM (#27432785)
It's interesting because Varnish gets it write in their docs: https://varnish-cache.org/docs/trunk/users-guide/troubleshoo...
Re:Fastly Outage by qeternity on Tuesday June 08, 2021 @06:24AM (#27432884)
> It's interesting because Varnish gets it write in their docs

Sorry I couldn't help myself...it's too funny to misspell "right" in a thread about spelling mistakes

Re:Fastly Outage by ljm on Tuesday June 08, 2021 @06:32AM (#27432984)
Mruphy's Law in action
Re:Fastly Outage by cromulent on Tuesday June 08, 2021 @06:43AM (#27433151)
Isn't it Muphry's Law? Or was that another example :)

https://en.wikipedia.org/wiki/Muphry%27s_law

Re:Fastly Outage by eecc on Tuesday June 08, 2021 @06:58AM (#27433391)
this is devolving quickly into an r/something
Re:Fastly Outage by saagarjha on Tuesday June 08, 2021 @06:26AM (#27432897)
;)
Re:Fastly Outage by dolmen on Wednesday June 09, 2021 @04:42AM (#27445637)
Re:Fastly Outage by nomilk on Tuesday June 08, 2021 @06:17AM (#27432749)
Same 503 error message on the actual http://fastly.com/ website
Re:Fastly Outage by zkldi on Tuesday June 08, 2021 @06:18AM (#27432768)
Mediation is a real word though?
Re:Fastly Outage by smcl on Tuesday June 08, 2021 @06:19AM (#27432800)
That is true, but the typo is because they presumably meant to reference this: https://en.wikipedia.org/wiki/Guru_Meditation
Re:Fastly Outage by Jenk on Tuesday June 08, 2021 @07:01AM (#27433427)
...and naturally someone has already updated the page today to include (and highlight) its use and mispelling on Varnish.
Re:Fastly Outage by richrichardsson on Tuesday June 08, 2021 @07:51AM (#27433885)
Someone (I can't unfortunately due to IP block) needs to change that. The part about the spelling is false, apparently [1] it's an intentional change by Fastly so that they can tell if it's their own Varnish or a customer's Varnish that is throwing an error.

[1] https://news.ycombinator.com/item?id=27433139

Re:Fastly Outage by nkozyra on Tuesday June 08, 2021 @06:55AM (#27433362)
That wiki references the Varnish version.

I think it's an intentional alteration of the original, given the context.

Re:Fastly Outage by Jenk on Tuesday June 08, 2021 @07:01AM (#27433434)
That edit was added after today's fastly outage began.
Re:Fastly Outage by nkozyra on Tuesday June 08, 2021 @07:07AM (#27433485)
Even so, mediation makes sense in this context given fastly's business model.

It could be a typo or an attempt to be clever.

Re:Fastly Outage by nkozyra on Tuesday June 08, 2021 @06:56AM (#27433374)
This seems like it's intentional given the context.
Re:Fastly Outage by an0n4u on Tuesday June 08, 2021 @06:16AM (#27432740)
I don't see the typo? definitely a fastly problem though..

Fastly error: unknown domain: numpy.org.

Details: cache-pdk17841-PDK

Re:Fastly Outage by sossles on Tuesday June 08, 2021 @06:18AM (#27432778)
I assume "Guru Mediation" is supposed to be "Guru Meditation", a reference to the way AmigaOS used to describe system failures.
Re:Fastly Outage by dijit on Tuesday June 08, 2021 @06:18AM (#27432767)
Mediation vs Meditation I think.

I think Varnish uses mediation intentionally though, it was this way 7 years ago when I last used Varnish.

Re:Fastly Outage by the_mitsuhiko on Tuesday June 08, 2021 @06:22AM (#27432847)
Re:Fastly Outage by patentatt on Tuesday June 08, 2021 @06:43AM (#27433149)
According to the Wikipedia page, it is intentional:

https://en.wikipedia.org/wiki/Guru_Meditation

Or did one of you already edit the Wikipedia page to reflect this discussion on hn?

Re:Fastly Outage by _rf on Tuesday June 08, 2021 @07:04AM (#27433462)
It appears the mention of "Mediation" was made very recently, likely in response to what's currently going on.

https://en.wikipedia.org/w/index.php?title=Guru_Meditation&d...

And it seems to be incorrect, since this "spelling variation" is only used by Fastly and not part of Varnish?...

Re:Fastly Outage by montag on Tuesday June 08, 2021 @06:17AM (#27432759)
The typo is "Mediation" instead of "Meditation", I think.
Re:Fastly Outage by eutectic on Tuesday June 08, 2021 @06:34AM (#27433011)
I assumed it was an intentional pun.
Re:Fastly Outage by c-fe on Tuesday June 08, 2021 @06:03AM (#27432510)
also https://www.reddit.com (at least in Netherlands)

edit: 12:05 up again for me, no images or custom fonts loading though ... and down again 1 minute later

edit: 13:01 reliably up again for me

Re:Fastly Outage by freebuju on Tuesday June 08, 2021 @06:34AM (#27433022)
Down in SA
Re:Fastly Outage by secondcoming on Tuesday June 08, 2021 @06:10AM (#27432645)
Down in UK
Re:Fastly Outage by daniaal on Tuesday June 08, 2021 @06:30AM (#27432958)
Up for me but no images mate.
Re:Fastly Outage by tacticalmook on Tuesday June 08, 2021 @06:15AM (#27432726)
Down in US. Also Imgur, which is closely related
Re:Fastly Outage by floating_panda on Tuesday June 08, 2021 @06:17AM (#27432757)
Down in india
Re:Fastly Outage by silviot on Tuesday June 08, 2021 @06:20AM (#27432812)
Same here in Germany: imgur and reddit are down, plus a bunch of other sites.
Re:Fastly Outage by tarsinge on Tuesday June 08, 2021 @06:22AM (#27432846)
Same in France
Re:Fastly Outage by kevincox on Tuesday June 08, 2021 @06:38AM (#27433074)
> potential impact to performance

So it is a "performance" issue when all pages give a 503.

Re:Fastly Outage by coolspot on Tuesday June 08, 2021 @02:22PM (#27438878)
Does the 503 page load fast(ly) or slowly?
Re:Fastly Outage by threeseed on Tuesday June 08, 2021 @06:06AM (#27432568)
I wonder why Amazon is not using Cloudfront for their own website.
Re:Fastly Outage by lordpankake on Tuesday June 08, 2021 @06:41AM (#27433108)
Cloudfront, by Amazon's own admission, specialises in high bandwidth delivery (ie huge videos). Fastly has consistently better performance as a small object cache, which makes it the choice for web assets

https://www.streamingmediablog.com/2020/05/fastly-amazon-hom...

Re:Fastly Outage by morelikeborelax on Tuesday June 08, 2021 @06:13AM (#27432701)
Fastly gives them the edge performance they need without having to build it themselves. They have been a customer for a while I think.
Re:Fastly Outage by raphaelj on Tuesday June 08, 2021 @06:14AM (#27432705)
But they have competing products through AWS.
Re:Fastly Outage by playpause on Tuesday June 08, 2021 @09:06AM (#27434615)
I imagine it works well for the whole business that they allow product teams to use the best cloud tools for the job rather than requiring them to use AWS for everything. If AWS is forced to compete even for Amazon.com's custom, that should make the whole company more resilient to long term technical stagnation.
Re:Fastly Outage by threeseed on Tuesday June 08, 2021 @06:16AM (#27432742)
AWS Route53 and Cloudfront are direct competitors to Fastly.
Re:Fastly Outage by morelikeborelax on Tuesday June 08, 2021 @06:39AM (#27433077)
That's how good Fastly is. Outside of this issue it's a great service.
Re:Fastly Outage by schappim on Tuesday June 08, 2021 @06:07AM (#27432594)
Yeah, this is what makes me feel this is more an AWS thing
Re:Fastly Outage by lpmitchell on Tuesday June 08, 2021 @06:11AM (#27432663)
The m.media-amazon.com domain (and a few other CDN'd domains that they use) are running through Fastly:

    nslookup m.media-amazon.com
    
    Name:  media.amazon.map.fastly.net

It is very interesting that they are not using CloudFront!
Re:Fastly Outage by cowsandmilk on Tuesday June 08, 2021 @06:50AM (#27433286)
really, m.media-amazon.com seems to have a very short TTL (showing 37 seconds right now) and has been weighted to cloudfront now.

Amazon is also known to use Akamai. Sure, Amazon relies heavily on AWS, but why should it surprise anyone that a retail website obsessed with instant loading of pages decides to use non-AWS CDNs if the performance is better.

Even if CloudFront became the default, I'm certain amazon.com would keep contracts with fastly and akamai just so they can weight traffic away from CloudFront in an outage.

Re:Fastly Outage by jamespo on Tuesday June 08, 2021 @06:38AM (#27433065)
Good to have 3rd party redundancy, time to fail over to something else now I'd think though.
Re:Fastly Outage by longwave on Tuesday June 08, 2021 @06:47AM (#27433229)
They already have:

  $ host m.media-amazon.com
  m.media-amazon.com is an alias for c.media-amazon.com.
  c.media-amazon.com has address 99.86.119.84
(which is a Cloudfront IP)
Re:Fastly Outage by xmdx on Tuesday June 08, 2021 @06:53AM (#27433330)
Yep they did exactly this and are now running on cloudfront
Re:Fastly Outage by tester34 on Tuesday June 08, 2021 @06:34AM (#27433018)
Why?
Re:Fastly Outage by adzicg on Tuesday June 08, 2021 @06:23AM (#27432873)
looks like amazon.com started using fastly in May 2020 (https://www.streamingmediablog.com/2020/05/fastly-amazon-hom...) so it's not an AWS thing
Re:Fastly Outage by threeseed on Tuesday June 08, 2021 @06:11AM (#27432673)
AWS is reporting no issues across the regions:

https://status.aws.amazon.com

Re:Fastly Outage by speedgoose on Tuesday June 08, 2021 @06:17AM (#27432765)
AWS is almost never reporting issues on this page.
Re:Fastly Outage by object_Object on Tuesday June 08, 2021 @06:46AM (#27433206)
AWS don't report outages until it's undoubtedly them.
Re:Fastly Outage by alex_duf on Tuesday June 08, 2021 @06:29AM (#27432952)
Fastly deploy their own hardware, (That's one of their selling points) I don't think they rely much on AWS, maybe just for network interconnection?
Re:Fastly Outage by lordpankake on Tuesday June 08, 2021 @06:41AM (#27433125)
Fastly doesn't run on AWS.
Re:Fastly Outage by apexalpha on Tuesday June 08, 2021 @06:08AM (#27432610)
It sure looks like a AWS error, even Amazon.com is mostly down.
Re:Fastly Outage by antihero on Tuesday June 08, 2021 @06:29AM (#27432946)
I wonder why amazon.co.uk uses Fastly and not CloudFront?
Re:Fastly Outage by georgekeesjones on Tuesday June 08, 2021 @06:45AM (#27433181)
I imagine they use a few different CDNs for things like this.
Re:Fastly Outage by abunuwas on Tuesday June 08, 2021 @06:40AM (#27433095)
That doesn't appear to be the case does it? Amazon sites are all working fine, at least for me
Re:Fastly Outage by danso on Tuesday June 08, 2021 @06:44AM (#27433180)
Their CSS and JS were down for a few minutes. I was able to login to Amazon but the entire site was in Times New Roman, but was fixed a few minutes later
Re:Fastly Outage by jonplackett on Tuesday June 08, 2021 @06:39AM (#27433079)
Must be more than fastly. Heroku is also down.
Re:Fastly Outage by abunuwas on Tuesday June 08, 2021 @06:42AM (#27433128)
That's also because of fastly, I've got this response from the Heroku dashboard:

Fastly error: unknown domain: dashboard.heroku.com.

Re:Fastly Outage by lkbm on Tuesday June 08, 2021 @10:17AM (#27435502)
Good thing we use Cloudfront and Cloudflare where I work.

> Statuspage Automation updated third-party component Spreedly Core from Operational to Major Outage.

> Statuspage Automation updated third-party component Filestack API from Operational to Degraded Performance.

Oh, right. :-D

Don't get me wrong, I love the proliferation of APIs and easily-integrated services over the past 20 years. We're all one interdependent family, for better and for worse.

Re:Fastly Outage by huijzer on Tuesday June 08, 2021 @06:20AM (#27432819)
CSS/Javascript at https://github.com/ appears to be down as well making GitHub quite unusable.
Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:34AM (#27433017)
GitHub Pages appears to be down too, taking an awful lot of sites offline
Re:Fastly Outage by busymom0 on Tuesday June 08, 2021 @06:44AM (#27433176)
Github is working fine for me in Canada but others aren't. Tried without browser cache too and it works okay.

EDIT: Most sites seem fixed now here in Canada. Tested stackoverflow, reddit, GitHub, PayPal, gov.UK and all worked fine.

Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:53AM (#27433334)
Yeah, things are mostly back now. Including my personal GitHub Pages site :)
Re:Fastly Outage by weird-eye-issue on Tuesday June 08, 2021 @06:01AM (#27432486)
Yikes seeing just a "connection failure" on Paypal is something else.

edit: PayPal looks be back up at least in US East but when I turn off my VPN and access from Asia I get "Fastly error: unknown domain: www.paypal.com."

Now I'm seeing a 503

Re:Fastly Outage by 1ncorrect on Tuesday June 08, 2021 @07:03AM (#27433447)
> Monitoring The issue has been identified and a fix has been applied. Customers may experience increased origin load as global services return. Posted 4 minutes ago. Jun 08, 2021 - 10:57 UTC

Looks to be working again my end.

Re:Fastly Outage by thrdbndndn on Tuesday June 08, 2021 @06:02AM (#27432493)
Interestingly, Twitter only has its emoji SVGs down.
Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:33AM (#27433003)
And this is (one reason) why using images instead of actual emojis is such a stupid idea. Why, Twitter, WHY?
Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @06:47AM (#27433231)
err, to make representations platform independent?
Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:54AM (#27433338)
That sounds antithetical to the purpose of emojis.
Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @07:18AM (#27433594)
Vendors don’t even agree on whether the :gun: is a revolver or an automatic or space ray guns or even water guns, btw it’s an 1911 in original DoCoMo emojis

1: https://blog.emojipedia.org/content/images/2018/04/microsoft...

Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @07:30AM (#27433702)
Sure, that's a benefit of emojis being semantic. If you want 'SFW' emojis, you can get them. Converting them to images makes that impossible. And uses vastly more bandwidth, makes them impossible to copy+paste, probably has accessibility issues, etc.
Re:Fastly Outage by mbreese on Tuesday June 08, 2021 @07:14AM (#27433564)
Same reason why Gmail uses their own emojis rather than the system ones — (as said above) branding. When you send a tweet, Twitter wants it to look identical across all devices. The classic native UI vs cross-platform UI debate in a nutshell.
Re:Fastly Outage by gspr on Tuesday June 08, 2021 @07:43AM (#27433820)
Cool, so instead of actually serving text, they could also just serve up little SVGs for each letter. Because god forbid the recipient chooses a different font than Gmail!
Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @08:01AM (#27433963)
Which is why Slack style :tofu_on_fire: emoji notation is genius
Re:Fastly Outage by GranPC on Tuesday June 08, 2021 @08:19AM (#27434133)
You're pretty much defining webfonts!
Re:Fastly Outage by gspr on Tuesday June 08, 2021 @08:24AM (#27434188)
Indeed. Another abomination.
Re:Fastly Outage by dolmen on Wednesday June 09, 2021 @04:54AM (#27445733)
That's not a minor UI issue.

Twitter is a media between people. Removing emoji representation differences on user devices is a way to hopefully reduce misunderstandings between users.

Re:Fastly Outage by bb010g on Tuesday June 08, 2021 @06:39AM (#27433082)
Branding! (Fun fact: Hacker News strips emoji.)
Re:Fastly Outage by busymom0 on Tuesday June 08, 2021 @06:45AM (#27433196)
How does it strip it? Test:

Edit: You are right. It got rid of the emoji after Test.

Re:Fastly Outage by doublerabbit on Tuesday June 08, 2021 @06:11AM (#27432667)
https://deb.debian.org is down too which borked my installation.
Re:Fastly Outage by benrbray on Tuesday June 08, 2021 @06:12AM (#27432677)
Re:Fastly Outage by fgaz on Tuesday June 08, 2021 @07:12AM (#27433546)
The mirrors still work though, and cabal will just fall back to those
Re:Fastly Outage by jakub_g on Tuesday June 08, 2021 @06:36AM (#27433049)
https://www.bbc.com/news/technology-57399628

"A number of leading media websites are currently not working, including the Guardian, Financial Times, Independent and the New York Times."

Re:Fastly Outage by spzb on Tuesday June 08, 2021 @06:43AM (#27433155)
Not that the BBC are gloating that they're still up
Re:Fastly Outage by Symbiote on Tuesday June 08, 2021 @06:57AM (#27433382)
The BBC.com site was down for about 10-15 minutes.
Re:Fastly Outage by toaway on Tuesday June 08, 2021 @06:08AM (#27432612)
Re:Fastly Outage by aliasEli on Tuesday June 08, 2021 @07:21AM (#27433627)
What's far worse than half of the internet being down was that Hacker News also had problems. If I waited long enough on a comments page I got an error message. I don't quite understand what happened there. The communication between my system and HN must have been working otherwise I would never have gotten an error message, so it must have been some internal HN problem. But since HN should only need its own internal "database" to generate comment pages, I don't understand why it should be impacted by the Fastly problems.
Re:Fastly Outage by jzer0cool on Tuesday June 08, 2021 @08:46AM (#27434403)
I could not tell from the fastly status page. What caused the fault? Could anyone point to any past stories which may be of similar nature other than DDos?
Re:Fastly Outage by bartimus on Tuesday June 08, 2021 @08:51AM (#27434460)
Re:Fastly Outage by zhan_eg on Tuesday June 08, 2021 @06:04AM (#27432535)
Bitwarden is also down (the Web Vault, not the website).
Re:Fastly Outage by 0xbkt on Tuesday June 08, 2021 @06:09AM (#27432634)
I will never understand the meaning of putting CDN behind CDN.
Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @06:55AM (#27433359)
What makes sense in the world is what puts bacon on the table and not what actually make sense.
Re:Fastly Outage by dTal on Tuesday June 08, 2021 @06:48AM (#27433249)
Yo dawg, I heard you like CDNs...
Re:Fastly Outage by iso1631 on Tuesday June 08, 2021 @06:09AM (#27432621)
My self hosted bitwarden is fine, as are all my self hosted sites.
Re:Fastly Outage by stevenwliao on Tuesday June 08, 2021 @06:23AM (#27432860)
Seems to affect Target ( https://www.target.com/ ) and Reddit ( https://www.reddit.com/ ) as well.
Re:Fastly Outage by tandav on Tuesday June 08, 2021 @06:21AM (#27432833)
Re:Fastly Outage by abhiminator on Tuesday June 08, 2021 @06:05AM (#27432561)
PayPal seems to be working for me at the moment. Rest of the sites are 503s.
Re:Fastly Outage by rvz on Tuesday June 08, 2021 @06:12AM (#27432683)
Centralising everything™ and the whole internet goes down because of that.
Re:Fastly Outage by aembleton on Tuesday June 08, 2021 @06:16AM (#27432738)
and yet you're able to leave this comment.
Re:Fastly Outage by rvz on Tuesday June 08, 2021 @06:21AM (#27432827)
Because HN and those who use less or those who use backup services are smart and those who are caught now have to panic and wait.

Probably going to short the hell out of $FSLY.

Re:Fastly Outage by morelikeborelax on Tuesday June 08, 2021 @06:30AM (#27432956)
Over one issue that highlights they have an abundance of top level customers? Interesting strategy when it's already at a low.
Re:Fastly Outage by rvz on Tuesday June 08, 2021 @06:42AM (#27433135)
One issue that should have been mitigated at least by Fastly; worse if the client has to do it.

They proudly stated this from their own website to their customers:

> "Fastly’s network has built-in redundancies and automatic failover routing to ensure optimal performance and uptime."

If that isn't one huge lie, I don't know what is.

Re:Fastly Outage by aliasEli on Tuesday June 08, 2021 @07:36AM (#27433755)
Please don't call it a lie. It means that they knowingly presented something they knew to be false as the truth. So far I have seen no evidence to support that.
Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:35AM (#27434292)
It is definitely a lie, but it's the same lie sold by all cloud offerings. Can you name a single cloud/CDN operator without downtimes?

It's normal to have downtimes but they are usually scheduled and quick (think <10 minutes per month for rebooting and/or hardware parts replacement). I'm pretty sure most non-profit hosts like disroot.org or globenet.org have similar or better 9's than all these fancy cloud services.

Re:Fastly Outage by detaro on Tuesday June 08, 2021 @08:37AM (#27434318)
It can have all these things and still fail, suggesting otherwise would be fairly naive.
Re:Fastly Outage by smhost on Tuesday June 08, 2021 @06:20AM (#27432806)
if by "everything" you mean one thing, and by "centralize" you mean not centralized, then sure.
Re:Fastly Outage by darkwater on Tuesday June 08, 2021 @06:33AM (#27433006)
How is having a large chunk of the internet using the same CDN provider not "centralizing"? It's not a hard monopoly obviously but still it meets the definition of centralization.
Re:Fastly Outage by smhost on Tuesday June 08, 2021 @06:45AM (#27433193)
how is private companies choosing to use a common supplier in a competitive market centralization? monopolies are not centralization either. you need to read a better book.
Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:37AM (#27434311)
How is a market competitive when there's a quasi-monopoly on infrastructure? When public money is used to irrigate the same corporations with huge $$$, while non-profit network operators are left to rot?
Re:Fastly Outage by darkwater on Tuesday June 08, 2021 @07:01AM (#27433426)
it's centralization because they all use the same provider. Why do you care about incentives here? The result is the same, just like capitalism and free market tend to monopolies in the long run.
Re:Fastly Outage by Simran-B on Tuesday June 08, 2021 @07:18AM (#27433602)
Re:Fastly Outage by jb1991 on Tuesday June 08, 2021 @06:40AM (#27433096)
For what its worth, I'm having these problems also with cnn.com, reddit and many others, however when I switch away from WiFi to use my cell provider network, they work fine.
Re:Fastly Outage by tus89 on Tuesday June 08, 2021 @06:14AM (#27432709)
Paypal back, off fastly
Re:Fastly Outage by jslakro on Tuesday June 08, 2021 @06:35AM (#27433033)
Why no other sites bypass the CDN and go directly?
Re:Fastly Outage by meibo on Tuesday June 08, 2021 @06:40AM (#27433091)
If you aren't prepared to do CDN changes on a whim when something like this happens, it's often better to wait for the problem to be resolved instead of making things worse for yourself due to misconfigurations, revealing your origin IPs, etc.

Can always improve the process for the next outage.

Re:Fastly Outage by jslakro on Tuesday June 08, 2021 @06:45AM (#27433191)
For sure, similar to other industries all changes come after big troubles like this. But would be interesting to heard about how them (paypal) deal with that
Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @07:20AM (#27433610)
Also it takes time for DNS changes to propagate(some people hates this word but actually)
Re:Fastly Outage by tus89 on Tuesday June 08, 2021 @06:40AM (#27433097)
You need big infra and Crack teams of ops people, which paypal can't afford not to have.
Re:Fastly Outage by dan-robertson on Tuesday June 08, 2021 @07:25AM (#27433660)
Is their anything these big sites could do in this situation, or must they choose between running and maintaining all of their own infra or relying on a single CDN?
Re:Fastly Outage by darkr on Tuesday June 08, 2021 @07:31AM (#27433710)
If you have absolutely vanilla CDN requirements, you can run multiple CDNs and fail-over or load balance between them using DNS.

Quite a few Fastly customers have more than vanilla requirements though, and may have a lot of business logic performed within the CDN itself. That Fastly is "just Varnish" and you can perform powerful traffic manipulation is one of it's main selling points.

Re:Fastly Outage by dan-robertson on Tuesday June 08, 2021 @01:30PM (#27438221)
I suppose it’s still a bad experience for the user if some % of attempts to connect fail or if some % of scripts/styles/images fail to load. So I think that means dns information about failures needs to somehow be propagated quickly. Not sure how well that works in practice.
Re:Fastly Outage by fmajid on Tuesday June 08, 2021 @07:51AM (#27433884)
Use two CDNs and DNS providers for redundancy. Gets expensive, but at scale, probably doesn't make a huge difference. More complexity for the site operators to manage, however.
Re:Fastly Outage by zarker on Tuesday June 08, 2021 @06:28AM (#27432934)
Spotify is behaving strangely as well https://www.spotify.com/
Re:Fastly Outage by mondaygreens on Tuesday June 08, 2021 @06:03AM (#27432517)
Quora and reddit too
Re:Fastly Outage by black_puppydog on Tuesday June 08, 2021 @06:02AM (#27432494)
All of these work from here in Grenoble, France...
Re:Fastly Outage by southerntofu on Tuesday June 08, 2021 @08:31AM (#27434259)
That's the problem with these black-box cloud offerings, that you can never know what will work (or not) and from where. You get semi-random, pseudo-localized outages that are not accounted for in all the 9's of availability.

With a standard TCP/UDP session, it mostly just works or doesn't and you can get a proper traceroute to know what's up. With these fancy CDNs, there's a whole new can of worms to deal with and from a client's perspective you have no clue what's happening because it's all taking place in their private network space where we have no "looking glass".

Fuck the cloud, i want real Internet.

Re:Fastly Outage by pixl97 on Tuesday June 08, 2021 @08:48AM (#27434435)
"Gets blasted by a DDOS and is no longer on the internet"
Re:Fastly Outage by terramex on Tuesday June 08, 2021 @06:08AM (#27432607)
Same here in central Poland (Łódź area), no problem with any of linked websites.

edit: My whole Twitter timeline is full of posts saying "Twitter outage? what outage?". Same on Reddit and Twitch chat, feels like for a short time I was invited into some exclusive circle lmao. StackOverflow and other StackExchange sites also work so I can look stuff up for you.

Re:Fastly Outage by clydethefrog on Tuesday June 08, 2021 @06:02AM (#27432507)
Interesting. Here in the Netherlands they don't.
Re:Fastly Outage by ju-st on Tuesday June 08, 2021 @06:11AM (#27432668)
Germany here (n=1), everything works except reddit and ft.com
Re:Fastly Outage by lutoma on Tuesday June 08, 2021 @06:38AM (#27433073)
Same from Germany, all of these seem up except for reddit and ft. Maybe we got lucky with our edge node...
Re:Fastly Outage by hengheng on Tuesday June 08, 2021 @06:37AM (#27433053)
Not in East Germany :D
Re:Fastly Outage by mckirk on Tuesday June 08, 2021 @06:53AM (#27433324)
What about banana.ch?
Re:Fastly Outage by bsaul on Tuesday June 08, 2021 @06:03AM (#27432520)
depends where in France, most people i know here are affected as well
Re:Fastly Outage by black_puppydog on Tuesday June 08, 2021 @06:04AM (#27432537)
yeah grenoble, updated.
Re:Fastly Outage by bambax on Tuesday June 08, 2021 @06:16AM (#27432744)
Not from Paris, France.
Re:Fastly Outage by agumonkey on Tuesday June 08, 2021 @06:02AM (#27432508)
Not for me
Re:Fastly Outage by EmptyStatement on Tuesday June 08, 2021 @06:24AM (#27432879)
Re:Fastly Outage by amiga-workbench on Tuesday June 08, 2021 @06:01AM (#27432467)
https://www.theverge.com/ seems to be down too
Re:Fastly Outage by conradfr on Tuesday June 08, 2021 @06:29AM (#27432943)
Re:Fastly Outage by cellover on Tuesday June 08, 2021 @06:45AM (#27433203)
Is the fact of looking at those links is like looking at a road accident with insistence instead of just passing by?
Re:Fastly Outage by zthxxx on Tuesday June 08, 2021 @06:19AM (#27432795)
Re:Fastly Outage by alvis on Tuesday June 08, 2021 @06:21AM (#27432832)
Re:Fastly Outage by xmdx on Tuesday June 08, 2021 @06:06AM (#27432574)
Terraform having issues and rubygems down too
Re:Fastly Outage by meepmorp on Tuesday June 08, 2021 @07:02AM (#27433440)
That explains the spotty container build failures over the last half hour. Good thing I decided to procrastinate instead of debugging the issue!
Re:Fastly Outage by max23_ on Tuesday June 08, 2021 @06:08AM (#27432605)
Re:Fastly Outage by sammygreen on Tuesday June 08, 2021 @06:04AM (#27432546)
Seems to be every site that runs varnish...
Re:Fastly Outage by meowface on Tuesday June 08, 2021 @06:13AM (#27432700)
Fastly largely runs on Varnish, it seems: https://www.fastly.com/blog/benefits-using-varnish

>At the core of Fastly is Varnish, an open source web accelerator that’s designed for high-performance content delivery. Varnish is the key to being able to accelerate dynamic content, APIs, and logic at the edge.

Re:Fastly Outage by busymom0 on Tuesday June 08, 2021 @06:49AM (#27433275)
I think Fastly is the one having problems (they happen to use varnish but I haven't seen anything which says varnish is the root cause) - so all sites using it are down.
Re:Fastly Outage by dreamer7 on Tuesday June 08, 2021 @07:22AM (#27433634)
Firebase hosting has been affected as well
Re:Fastly Outage by playpause on Tuesday June 08, 2021 @06:05AM (#27432563)
Re:Fastly Outage by rottc0dd on Tuesday June 08, 2021 @07:12AM (#27433544)
SSO and github are back online now
Re:Fastly Outage by maelito on Tuesday June 08, 2021 @06:23AM (#27432864)
nature.com
Re:Fastly Outage by linuxfan2021 on Tuesday June 08, 2021 @09:02AM (#27434577)
You would think that the UK GOVERNMENT would have their own private CDN or something...
Re:Fastly Outage by iso1210 on Tuesday June 08, 2021 @09:26AM (#27434844)
Why?
Re:Fastly Outage by keithnz on Tuesday June 08, 2021 @06:27AM (#27432910)
twitch also, lots of other minor ish websites
Re:Fastly Outage by brador on Tuesday June 08, 2021 @06:04AM (#27432536)
Searchable offline backup of stack anyone?
Re:Fastly Outage by toaway on Tuesday June 08, 2021 @06:34AM (#27433010)
www.gov.uk & bbc are back
Re:Fastly Outage by dpacmittal on Tuesday June 08, 2021 @06:22AM (#27432845)
elastic.co down as well
Re:Fastly Outage by treeshateorcs on Tuesday June 08, 2021 @06:21AM (#27432835)
developer.spotify.com
Re:Fastly Outage by magicturtle on Tuesday June 08, 2021 @06:04AM (#27432533)
reddit down aswell
Re:Fastly Outage by bartread on Tuesday June 08, 2021 @06:50AM (#27433294)
It's OK though, because large swathes of this discussion seem to have turned HN into reddit, at least temporarily. Normal service will no doubt resume in due course.
Re:Fastly Outage by ramshanker on Tuesday June 08, 2021 @06:35AM (#27433039)
twitch.tv Too.
Re:Fastly Outage by Hani1337 on Tuesday June 08, 2021 @06:10AM (#27432650)
etsy.com too
Re:Fastly Outage by madeofpalk on Tuesday June 08, 2021 @06:36AM (#27433045)
> [0] https://www.gov.uk/

Just checked, thank god the NHS vaccine site is still available - vaccines just got rolled out for under 30s today.

Re:Fastly Outage by patentatt on Tuesday June 08, 2021 @06:41AM (#27433113)
Edit: I didn’t mean anything negative here! Just slightly shocked that as the UK is opening up under 30 vaccinations, the US is struggling to find any more willing takers. It’s really probably a sign that there’s fewer anti-vaxxers in the UK more than anything. And that universal healthcare is more efficient at distribution than an inherently for profit system. I don’t know, but I just didn’t realize it was so different in the UK
Re:Fastly Outage by bdsa on Tuesday June 08, 2021 @06:46AM (#27433213)
I think this may be because we've had much higher uptake as far as I know, so getting down the age ranges has been slower (by which I mean, yes, maybe the US has made it available to all adults, but how many (as a proportion) have taken it up)
Re:Fastly Outage by patentatt on Tuesday June 08, 2021 @07:12AM (#27433545)
This is awesome. And also exposes how broken the US is with its anti-vaccination trend
Re:Fastly Outage by bdsa on Tuesday June 08, 2021 @08:44AM (#27434391)
I have seen the argument made that one of the reasons for high vaccine confidence in the UK is as a result of Andrew Wakefield's MMR fraud, which was perhaps debunked more effectively in the UK than the US.

https://www.youtube.com/watch?v=8BIcAZxFfrc

Re:Fastly Outage by madeofpalk on Tuesday June 08, 2021 @07:27AM (#27433670)
US and UK have very similar vaccination rates despite the US being open to more age ranges. This indicates that a higher percentage of eligible people have gotten the vaccine in the UK, and the US has somewhat hit a wall in terms of vaccinations (though there is the concern that the rates will slow down in the UK also).

I must admit, it has been strange seeing my US peers getting the vaccine months before I can in the UK, but I guess I take comfort knowing that both countries are still doing pretty well!

Re:Fastly Outage by numpad0 on Tuesday June 08, 2021 @07:05AM (#27433467)
You know which one’s worst? Japan... still reservation based and for 65 and up only!
Re:Fastly Outage by zhte415 on Tuesday June 08, 2021 @07:23AM (#27433641)
Both the UK and US are doing well.

https://ig.ft.com/coronavirus-vaccine-tracker for reference.

What's important is important to share vaccines with all nations, and non-nations.

Re:Fastly Outage by spzb on Tuesday June 08, 2021 @06:48AM (#27433246)
Re:Fastly Outage by patentatt on Tuesday June 08, 2021 @07:10AM (#27433529)
Fascinating. So those rates are including only ages 30+, which means that once it’s unrestricted the UK should have a very high vaccination rate while ~15-25% of the US will still remain unvaccinated entirely by choice. Wow. So you’re absolutely right, the UK is in reality far far ahead and the US is completely broken as far as public health is concerned because of willing ignorance.
Re:Fastly Outage by oneeyedpigeon on Tuesday June 08, 2021 @06:56AM (#27433370)
For one dose. For full vaccination, the US is (slightly) ahead according to that same site.
Re:Fastly Outage by spzb on Tuesday June 08, 2021 @07:01AM (#27433428)
I think we can agree it's certainly not "far behind"
Re:Fastly Outage by lol768 on Tuesday June 08, 2021 @07:05AM (#27433471)
This is by design though, the gap between the two doses is higher here.
Re:Fastly Outage by PaywallBuster on Tuesday June 08, 2021 @06:05AM (#27432562)
and Imgix
Re:Fastly Outage by collyw on Tuesday June 08, 2021 @06:47AM (#27433230)
Click the new tab. Lots of posts about sites being down. All flagged.
Re:Fastly Outage by milliams on Tuesday June 08, 2021 @07:10AM (#27433517)
Yes, because they're all just repeats of this one.