Well to play devil's advocate if docker is part of your infrastructure and you need to make more than 200 pulls in 6 hours you should already host your own registry and maybe mirror a few dockerhub repos to your own
It really sounds like a problem we just have to solve. Putting all the expectation on one central service is not reliable.. it should be a distributed network of content delivery nodes, of which you run a few yourself.
I like this idea. I would think that the “default” hub should not be managed only by Docker, but eg by docker and other registries. I suspect that docker inc still wants full control over their registry though so I don’t see this happening.
We might decide on another, non dockerhub open registry though, and use that instead.
Do pulls for a "latest" tag bypass the cache?
No, latest is just a tag and is not handled differently. There is a CLI flag to ignore cached layers. This however does not effect the FROM line, so if you have the image, there will be no pull.
Expect this to effect CI systems
Re:Docker to rate limit image pulls
by mikepurvis on Monday August 24, 2020 @02:09PM
(#24263521)
No they don't, but if you do something silly like spin up a bare VM build node, and then pull your environment every time, then obviously you won't get any caching.
Re:Docker to rate limit image pulls
by ArchOversight on Monday August 24, 2020 @02:26PM
(#24263731)
Github Actions? Gitlab CI? Many of those don't do any caching because of the chance of poisoning a tag in the local registry, and don't have a good way to do caching per project.
I can imagine that this affects those sorts of operations.
Yeah, that's what I was getting at - in CI I can see it, but in local dev it would seem surprising to need more than the free-with-authentication quota.
That said, if your CI needs more, it's probably time to invest in a paid account if you can, and certainly if you're commercial.
You can use the GitLab Dependency Proxy for caching images from DockerHub, it's pretty straightforward to pull cached images:
Using `docker-machine`
You can use an environment variable to pull from a local docker mirror from new bare VMs.
`export ENGINE_REGISTRY_MIRROR=https://mirror.mysite.com`
At some level this seems to me like using my IDE and after 6 hours it would stop working or finding that my CDNJS references to bootstrap stopped working after 6 hours of my site being up. I think it is exactly as if NPM or PIP were to stop working for the day if you included "too many" packages.
I don't really have a good feeling for how these new limits might impact setting up new dev/test environments so I'll likely switch from using specific images to using generic images to limit my exposure.
For example, at the moment, if I want a new dev container for a project in python, I'll FROM python:latest and supplement with pulls from support service containers like nginx:latest and postgres:latest.
Moving forward, it would seem a safer approach will be to use pull a single 18.04 image and run the required installs into them. This super bums me out though as it seems to bypass some of the nicer aspects of getting up and running with docker.