If you’re deploying to a cluster with 200 machines, you could easily hit this if you use the public registry though. However, if you’re managing that size cluster you can probably afford the fee, but more importantly, you should probably pull once to a local registry and use that to deploy to your cluster anyway.

If you do something like this, you absolutely MUST have a local registry.

Harbor [1], JFrog [2], and Quay [3] would be the first ones that I look at.

Harbor is open source, free, and a member of the CNCF. You will need to do a little bit of work to set it up to scale properly. JFrog offers a SaaS registry, but you will pay big $$ based on pull traffic. Their commercial site license is about $3k/year. Quay is older than either of them, stable, and high quality. I'd start with Harbor these days.

[1] https://goharbor.io/ [2] https://www.jfrog.com/confluence/display/JFROG/JFrog+Artifac... [3] https://quay.io/

I have pulled and run 20k times a 1GB image in less than 10-15 minutes without breaking a sweat.

Finally GitHub packages offers a registry out of the box . It is great for CI and devs to access . I generally have the tags mirrored from tags GitHub for production to ACR .

1) It is broken and unusable on Kubernetes and Docker Swarm.

2) It is flaky often returning 500 type errors.

3) It is expensive as the amount of pull bandwidth is very limited.

Hmm, I use them on several kubernetes clusters in the past few months and don't see any issue yet.

The main issue was ECR has a slightly different authentication model than docker swarm. The whole '--with-registry-auth' only partially works when you are using ECR. Unfortunately, it works just enough that you think it's working, until all your tokens time out and a worker can suddenly no longer pull an image.

Our common failure case was an image becoming unhealthy or a node being drained. When that image would try to be restarted on a different worker, if that worker did not have the image it would try to get it from the registry. If the tokens were expired it would fail.

The only "fix" we ever found was to setup a cron job that forcibly deployed a new version of a "replicated globally" image every X minutes (where X was based on ECR token expiration). It kind of worked, but we still had occasional failures we could not identify.

I wish it worked better, because it was nice to use ECR. Frankly token expiration sounds much more secure too, but without direct support for token refresh inside the docker engine it's just hard to get everything to work

That said, word of warning for anyone looking at GitHub Packages for docker registry: it's broken with containerd and some other similar tools. They (GitHub) are currently working on a fix: https://github.com/containerd/containerd/issues/3291

It's not transparent though.

With ECR you pay for image storage: $0.09 per GB after the first 1 GB which is free

[1] https://aws.amazon.com/ecr/

pull from docker hub once, push to ECR. then pull from ECR as much as wish

you can set it up in less 10 min and the only thing required is to add '--insecure-registry' in your client. It is not a issue if all your machine are in private network.

still wonder how to do it in minutes.

Expect to spend 1-2 hours first time you try it until you can setup the correct DNS records, API keys and configuration.

Afterwards it's pretty hands off, every three months you'll receive an email from letsencrypt and you'll have to rerun this script to regenerate your certificates. Takes 2-3 minutes max (but of course you still need to distribute your certificates to all relevant services...)

It's hard enough to save money as it is.

News websites blow my mind with this - if I forked over $5 to every news outlet I occasionally like to read, I'd be spending at least $500 maybe more per year JUST to get access to some random person's biased recant of what's happening in the world. If there were a news source that did the opposite of this, and basically provided a bullet list of objective, non-biased events boiled down to exactly what I need to know, that might be something I'd pay for. Hell, it would save you time over filtering the opinionated BS out.

Consider for example the current riots going on in the US. How do you objectively report on that? With bias, on one side you have "peaceful protest disrupted and escalated by the police", on the other side you have "police intervening in riots to maintain order and protect property". There's not really inbetween.

Done.

Much as we'd all love an "unbiased" news source, the reality is that bias is a very hard problem to solve well.

If one side is lying, objective reporting would tell you which side it was.

Also there are news wire services that do mostly what you're describing. If you just want to be entertained (most people read news for entertainment), then they don't really care about the facts. They want to hear about so and so blasting so and so or whatever. But if you're trying to make money from information (traders, journalists, etc), then you really don't want to be reading the kind of stuff the New York Times is publishing.

Just a subscription to The Information is $399/yr. Add a subscription to the Times, and you've already blown past your $500 budget.

If you get your news like most people, via link aggregators like HN, Reddit Facebook, Twitter then you get linked to dozens of publications that all want a $5/mo. commitment which is untenable.

But... in their news app, there's always a couple of interesting articles I might want to read, but I'm not signing up. They have my info, and a Touch ID device I'm holding tied to my payment info. "Read this article for 50c?" I'd certainly give some a read now and then.

I have been subscribing valuable information sources since 1995, so I do get the point.

The perpetual spam is worse for me than the $5.

It isn't about the money, it's about the Mommy-May-I up and down the chain with emails and meetings and careful explanations to skeptical glares. It's a psychological and institutional barrier.

Or has toxic micromanaged structure. I've had friends who have worked places that would barf over ongoing $60/year software charges, where anything like that would have to go up to C levels and require justification. Luckily never worked at one myself, dodged that particular bullet.

Handling each purchase and documenting it in case we are ever audited requires easily $25-50 of people's time.

> https://en.wikipedia.org/wiki/John_Thain

"Sorry I got caught. I will work harder to hide next time."

The headaches to spend $5 cost way more than $5.

this is where miracle of enterprise sales happen - the $5 subscription can be sold as a $50K+ deal by smooth enterprise sales who will provide the C-exec with the experience making him feel like he did something smart and great for the company.

Engineers can head this off by prepping a total-cost-to-execute analysis for mgmt. This doesn't need to be complicated. It's just some estimates of what's needed and why, and what the alternatives cost. My eng VP used to ask me for these when I'd send up a request. He wanted to know that we thought about total-cost-to-execute. He'd usually only read the exec summary and approve. If these requests are really going that far up the mgmt chain either someone isn't doing their job, or higher level mgmt are micromanagers.

I want to speed forward 5 years and see how well this ages. It reminds me of all the other comments about "of course Facebook will never require an account to use your VR headset"...

If it was $5 for access that would be a completely different situation, but a big free tier followed by $5 for unlimited is fine.

"Stock CentOS was good enough before and it's good enough now!"

I do sympathize with Docker though: Storage and bandwidth at that scale isn’t cheap and they need to monetize somehow.

The base costs for a lot of tech stuff (like bandwidth) are so cheap. But you would never know between these bullshit "what is it worth to you" pricing models and the number of middlemen trying to stick their hand into the pot. It's disgusting.

Note could, not will