We steer clear of proprietary cloud services. In place of S3, we use minio.
The one sore spot I haven’t been able to solve yet is interactive debugging. k8s pushes you toward a model where all services need a pile of environment variables to run, so setting these up is pretty painful. In practice, though, all rapid iteration happens inside unit tests, so not having interactive debugging isn’t much of a productivity drag.
Even on my personal Linux desktop, I don't love developing in containers. It is very tedious to context switch between my local environment and the in-container environment, and I don't even consider myself the type with a super personalized setup.
So I don't consider local docker that much of an improvement over a remote devbox.
I think, with my proposed setup, you'd still do development on your local machine. The containers would only be there for the dependencies, and as a shell to execute your code. The container hosting the application under development would use a volume to point to the files on your local machine, but the container itself (with all its permissions and configuration) would match or nearly match what you plan for the production environment.
1. New shared builds update container images for applications that comprise the environment
2. Rather than a "devbox", devs use something like Docker Compose to utilize the images locally. Presumably this would be configured identically to the proposed devbox, except with something like a volume pointing to local code.
I'm interested in learning more about this. It seems like a way to get things done locally without involving too many cloud services. Is this how most people do it?