I know what we have today is really auto complete 2.0 models that are really impressive good and bad but we are chasing that holy Grail as you mentioned. Star Trek specifically takes Data without emotions and later his brother lore who has emotions but is really the extreme of humanity and is basically as you say - uncontrollable.
I think it's important as society takes its first steps towards AI or similar we really consider this more.
Lore came before Data in the Trek lore (no pun intended), which also helps GP's argument - they went all the way, then when it misbehaved they dialled back to a less "perfect" but much simpler and more predictable solution.
I don't think regular software controls are flexible enough but there is research in making AI create its own "RLHF" fine-tuning dataset, like Constitutional AI from Anthropic. You only need to give it a few general guidelines and let it figure out the details. Never heard anyone complain Claude is unsafe or a bad chat experience.
If you have to encode the exact if-else conditions, then how it is different than regular software development.
So we want a system which we want it to behave in just about "perfect" manner (i.e. intelligent but not going astray). What passes as "perfect" is subjective and there will be different definitions of it.
Now to control this you want a capability to precisely define the allowed and dis-allowed behaviour. At this point we are essentially circling back to the software systems before the AI.