Pinyin is not a transliteration into English, it's a way to write Chinese using the latin alphabet. Learning how to pronounce Pinyin is no difference than learning how to pronounce French or German text.

So when you say "why the speaker is blamed rather than the transliteration" it's like saying "why the English speaker is blamed rather than the French spelling" about an English speaker having problems pronouncing a French word.