Yes, as long as the back end for the container supports it, RO sections of shared libraries will be shared and pulled from the same cache when available. The functionality that enables shared memory (and L* cache access in general) is implemented in silicon in the MMU so as long as the backend properly updates the page tables, you can share pages across any container or VM (except when prohibited by other virtualization hardware).

It's not something that happens automatically though because each kernel is responsible for telling the MMU how it should map memory for its child processes only. Any cross container page sharing has to be implemented at the host level where the kernel has unrestricted access to all guest memory.