News for hackers. Stuff that matters.
Re:Docker Systems Status: Full Service Disruption by KronisLV on Monday October 20, 2025 @04:22AM (#45641239)
I guess people who are running their own registries like Nexus and build their own container images from a common base image are feeling at least a bit more secure in their choice right now.

Wonder how many builds or redeployments this will break. Personally, nothing against Docker or Docker Hub of course, I find them to be useful.

Re:Docker Systems Status: Full Service Disruption by tom1337 on Monday October 20, 2025 @06:02AM (#45642093)
We are using base images but unfortunately some github actions are pulling docker images in their prepare phase - so while my application would build, I cannot deploy it because the CI/CD depends on dockerhub and you cannot change where these images are pulled from (so they cannot go through a pull-through cache)…
Re:Docker Systems Status: Full Service Disruption by roryirvine on Monday October 20, 2025 @08:48AM (#45643299)
My advice: document the issue, and use it to help justify spending time on removing those vestigial dependencies on Docker asap.

It's not just about reducing your exposure to third parties who you (presumably) don't have a contract with, it's also good mitigation against potential supply chain attacks - especially if you go as far as building the base images from scratch.

Re:Docker Systems Status: Full Service Disruption by tom1337 on Monday October 20, 2025 @08:51AM (#45643326)
Yea we have thought about that - I also want to remove most dependencies on externally imported actions on GitHub CI and probably just go back to simple bash scripts. Our actions are not that complicated and there is little benefit in using some external action to run ESLint than just run the command inside the action directly. Saves time and reduces dependencies - just need to get time to do that…
Re:Docker Systems Status: Full Service Disruption by enigmo on Monday October 20, 2025 @09:40AM (#45643797)
mirrors can be configured in dockerd or buildkit. if you can update the config (might need a self-hosted runner?) it’s a quick fix - see https://cloud.google.com/artifact-registry/docs/pull-cached-... for an example. aws and azure are similar.
Re:Docker Systems Status: Full Service Disruption by tom1337 on Monday October 20, 2025 @09:45AM (#45643860)
Hmm yea with a self hosted runner this could work. Gotta need to set the dockerd config into the VM before the runner starts I assume - unfortunately GitHub itself does not allow to change anything for the prepare stage - and it's a known issue for 2 years at least...

https://github.com/actions/runner-images/issues/1445#issueco... https://github.com/orgs/community/discussions/76636

Re:Docker Systems Status: Full Service Disruption by Sphax on Monday October 20, 2025 @04:39AM (#45641350)
We run Harbor and mirror every base image using its Proxy Cache feature, it's quite nice. We've had this setup for years now and while it works fine, Harbor has some rough edges.
Re:Docker Systems Status: Full Service Disruption by thephyber on Monday October 20, 2025 @06:17AM (#45642210)
I came here to mention that any non-trivial company depending on Docker images should look into a local proxy cache. It’s too much infra for a solo developer / tiny organization, but is a good hedge against DockerHub, GitHub repo, etc downtime and can run faster (less ingress transfer) if located in the same region as the rest of your infra.
Re:Docker Systems Status: Full Service Disruption by frenkel on Monday October 20, 2025 @04:29AM (#45641280)
Only if they get their base images from somewhere else...
Re:Docker Systems Status: Full Service Disruption by bravetraveler on Monday October 20, 2025 @05:16AM (#45641714)
Pull-through caches are still useful even when the upstream is down... assuming the image(s) were pulled recently. The HEAD to upstream will obviously fail [when checking currency], but the software is happy to serve what it has already pulled.

Depends on the implementation, of course: I'm speaking to 'distribution/distribution', the reference. Harbor or whatever else may behave differently, I have no idea.

Re:Docker Systems Status: Full Service Disruption by nusl on Monday October 20, 2025 @05:09AM (#45641641)
Currently unable to do much of anything new in dev/prod environments without manual workarounds. I'd imagine the impact is pretty massive.

Asside; seems Signal is also having issues. Damn.

Re:Docker Systems Status: Full Service Disruption by cebert on Monday October 20, 2025 @05:12AM (#45641683)
I’m not sure that the impact will be that big. Most organizations have their own mirrors for artifacts.
Re:Docker Systems Status: Full Service Disruption by VenturingVole on Monday October 20, 2025 @05:14AM (#45641699)
From what I've seen: I highly doubt it.

Edit to add: This might spur on a few more to start doing that, but people are quick to forget/prioritise other areas. If this keeps happening then it will change.

Re:Docker Systems Status: Full Service Disruption by nusl on Monday October 20, 2025 @05:19AM (#45641743)
Yeah, perhaps. I don't know how many folks host mirrors. Most places I've worked for didn't, though this is anecdotal.
Re:Docker Systems Status: Full Service Disruption by phillebaba on Monday October 20, 2025 @05:41AM (#45641922)
I would say most people would say it't best practice while a minority actually does it.
Re:Docker Systems Status: Full Service Disruption by CaptainOfCoit on Monday October 20, 2025 @06:14AM (#45642188)
Seems related to size and/or maturity if anything. I haven't seen any startups less than five year old doing anything like that, but I also haven't seen any huge enterprise not doing that, YMMV.
Re:Docker Systems Status: Full Service Disruption by walkabout on Monday October 20, 2025 @12:02PM (#45645449)
“Their own” can and often does mean something hosted on a major cloud provider (whether they manage it in-house or pay a vendor for their system)
Re:Docker Systems Status: Full Service Disruption by ai-onehealth on Monday October 20, 2025 @09:09AM (#45643476)
Yes I noticed Signal being down too
Re:Docker Systems Status: Full Service Disruption by jsmeaton on Monday October 20, 2025 @04:47AM (#45641426)
Guess where we host nexus..
Re:Docker Systems Status: Full Service Disruption by yandie on Monday October 20, 2025 @02:02PM (#45647055)
It's actually an important practice to have a docker image cache in the middle. You never know if an upstream image is purged randomly from docker, and your K8s node gets replaced, and now can't pull the base image for your service.

Just engineering hygiene IMO.

Re:Docker Systems Status: Full Service Disruption by koolba on Monday October 20, 2025 @09:35PM (#45651517)
> You never know if an upstream image is purged randomly from docker, and your K8s node gets replaced, and now can't pull the base image for your service.

That doesn’t make sense unless you have some oddball setup where k8s is building the images you’re running on the fly. Theres no such thing as “base image” for tasks running in k8s. There is just the image itself and its layers which may come from some other image.

But it’s not built by k8s. Its be built in whatever is building your images and storing I. Your registers. That’s where you need your true base image caching.

Re:Docker Systems Status: Full Service Disruption by yread on Monday October 20, 2025 @03:04PM (#45647839)
That is nothing compared to how good i feel about not using containers at all.
Re:Docker Systems Status: Full Service Disruption by bombcar on Monday October 20, 2025 @08:29PM (#45651097)
You don’t want a Rube Goldberg contraption doing everything?

So not agile!