Local caching is also a royal pain when one has multiple CI runners, besides the hassle of Docker related configuration.

Do they cache to shared storage and worry about concurrency? Do they only have local storage in various differing states?