But is it the case that the Yubikey is essentially treated the same as a trusted device? What if I want to untrust my devices and only trust ubikeys (without removing the device from my icloud account?)

I don’t seem to have the push option now

Yes but my understanding is that you can remove the Yubikey without possessing it, just with a “trusted device”. I want to mark all of my devices untrusted (wrt icloud account changes) and rely only on Yubikeys

From your apple doc:

“ When you use Security Keys for Apple ID, you’ll need a trusted device or a security key to:

Sign in with your Apple ID on a new device or on the web

Reset your Apple ID password or unlock your Apple ID

Add additional security keys or remove a security key”

Yubikeys do nothing except enlarge your attack surface.