You’ve identified the problem with many Bayesian approaches, not just this one. Without sufficient data to make the probabilities accurate, it just shifts the uncertainty to the process of choosing probabilities.