There is potentially bias (of the bad word variant) introduced by the mismatch between the prior in your own mind and the distribution and params you choose to try to approximate that, especially if you're trying to pick out a distribution with a nice posterior conjugate.

I'm also not sure why everyone perceived my comment as anti-bayesian.

I did not perceive your comment as anti-bayesian, or at least not necessarily so! :)

But are you sure you know what I meant when I said “uninformative prior”? Because choosing an uninformative prior does not involve choosing any parameters: there is only one uninformative prior, and it’s the constant (flat) distribution which assigns equal probability to every value. It encodes no information and does not bias the posterior or result. It is the one and only mathematically-neutral prior. You can think of it as being a bit like an “identity function”.

Depending on the hypothesis space what you call the "uninformative prior" does not exist in the frequentist approach. If you search for a real value, then the uninformative prior is a uniform distribution on the infinite line. This distribution does not normalize.

Ultimately, I think you are strawmanning frequentism here. Just because the log likelihood is sometimes the same as the map does not imply that they have the same meaning. This is why computed uncertainties of both approaches are often not the same and have a not-so-subtle difference in their interpretation. The one computes uncertainty in belief, the other imprecision of an experiment. You can't summarize that with "do you want to be explicit about assumptions".

Nobody normalizes an uninformative prior on its own. Normalization only happens when you get your posterior.

I am not straw-manning anything: bayesian methods are a generalization of frequentist methods. The equality / isomorphism to frequentist methods, in the special case of the uninformative prior, is commonly demonstrated in introductory (undergrad-level) bayesian textbooks and is in fact trivial. One need not even talk of any infinities: if you’re doing discrete observations, infinities never show up (and in that case the prior normalizes just fine). And if you’re curve-fitting (ie.: using parametric methods), the “infinite” line goes away as soon as you multiply your uninformative prior with your likelihood.