What I always wondered about is that this generally assumes that classes are assigned with an equal (human level) error probability. While this is certainly the case in heavily curated example dataset, many real world scenarios only consist of a considerably labeled positive set while the negative set is often drawn randomly from the background. Is there anything on how this can be taken into account (Besides weighting, obviously)?