The reference glyph for ' (code point 000027) <https://www.unicode.org/charts/PDF/U0000.pdf> shows a straight rendering. Any Unicode font rendering it differently can reasonably be said to be rendering it incorrectly.

This does not work, because the “right-skewed” apostrophe is not appropriate for every case of “straight” apostrophe placement.

However, making every “skewed” apostrophe “straight” is very easy for a computer to do (e.g. MusicBrainz Picard ships with a setting for this). This is why the MB guidelines say to prefer correct typographical apostrophes.

Note that they are preferred and not required, so you can still enter straight ones, but another editor may update them. I honestly don’t see the problem, but I am very pedantic about other things so I guess I can imagine how seeing the curlies could rub you the wrong way!