That's a good step forward. This separation is a massive barrier to entry. Making CUDA work reliably on every product was a masterstroke by NVIDIA.

I am getting the impression their (AMD) understanding of software/Devx is not as good as nvidia. There seems to be some sort of inertia in getting the software side moving - as if, now that they are behind, they are not willing to make any mistakes, so they hold off on doing anything decisive - instead opting for continuing the status quo mode but perhaps adding a few more resources.

The issue is that RDNA and CDNA are different so when an enthusiast makes a fast RDNA code it doesn't mean it would work well on CDNA and vice-versa. Not sure why AMD had to go this route, only high-end pros will write software for CDNA and they won't get any mindshare.

What do you have in mind for code that works noticeably better on one than the other?

That's not accurate at all.

Someone writing for the rocm platform will be writing HIP and then HIPCC will compile that down into the actual runtime targeting the given architecture. HIP is pretty platform agnostic so very little device specific optimization tends to go into idiomatic HIP.

There are many low-level details in CUDA to get it working fast on a given GPU architecture that compilers don't address at all.