it isnt particularly true, when other ML models have been shown to reinforce the ethics of the training data, eg. racism in hiring.

judgement is being added, and its hard to tell exactly what judgement it is and whether you agree or not

> tacit knowledge is not easy to add to the AI’s training set.

Is it even possible? Different people do different judgements for the same things, there is no right or wrong really, just the choice you made in that moment, based on whatever you cared about at that point.

Circumstantial tacit knowledge can probably be approximated through examples and feedback. But that is not quite the same as truly acquiring it. Bridging that gap would likely require a fundamental breakthrough. And sure, there is no objective right judgement, but that is the same for a lot of things the models already do.

This is the entire purpose of alignment, and we're very bad at it.