RLHF worked well for midjourney but I think that is because it is outsourcing something that is ultimately completely subjective and very subtle like human visual aesthetic choice that can't be "wrong".

I tried to understand the paper and I can't really make sense of it for "code".

It seems like this would inherit a subtler version of all the problems from expert systems.

A press release of this does feel rather AI bubbly. Not quite Why The Future Doesn't Need Us level but I think we are getting close.