OCI images are not quite as bad as they used to be. Linear caching is no longer baked directly into the image format, instead it's an assumption that has been carried forward by Dockerfiles and docker build.
Docker folks are working on at least the docker build part in buildkit[1]. In the meantime though I prefer Cloud Native Buildpacks, which are able to perform layer rebasing as an update operation.
Disclosure: I have previously worked on buildpacks technology for Pivotal, now VMware.
[0] https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU...
You might want to check out Garden for that (https://docs.garden.io/basics/how-garden-works). It thinks about your stack as a graph and manages build and deploy dependencies. And it supports the use case you're describing, i.e., you can combine a Docker image and a set of K8s manifests into a single "module" and the manifests can reference the Docker image.
Full disclosure: I'm affiliated with the project.
https://docs.earthly.dev/examples/monorepo
At least it addresses building the right Docker image--it doesn't seem to address building the broader Kubernetes manifests, but I'll take what I can get at this point.
[0] https://cnab.io/
The problem I see with the entire Docker ecosystem is that the Docker/Dockerfile build system is fundamentally terrible, principally because the cache system assumes a linear dependency tree and build graphs are, well, graphs. To get reasonable build performance, you need a tool that understands this. This is something Nix gets right, and it even lets you build Docker images without the Docker/Dockerfile build system.
I'd also like to see something that takes it a step further--build the docker image AND a set of kubernetes manifests that reference that docker image (a "kubernetes app package" if you will) such that you have a single artifact that represents your application that a "kubernetes package manager" can deploy (the current crop of k8s package managers altogether punt on the contents of docker images).
I understand that this is a very different way of thinking about Kubernetes and Docker builds than what we practice today, but I'd really like to see this in practice or hear some debate about its merits (or lackthereof).