Nope, it's based on the first sound. (Hence "an hour" rather than "a hour").

This is also why it's "a university".

Wait, what? So the rule isn't that the pronunciation starts with a vowel sound? Instead it's a more murky thing about how easy it rolls off the tongue?

University starts with the 'y' sound, hence "a university." On the other hand, "an unimpressive fact" uses "an" because "unimpressive" starts with the 'u' (vowel) sound.

University has a kind of "y" sound, like the word "yes", which isn't considered a vowel sound.

Doesn't "university" start with the "y" consonant sound?

Don't tell the knuckle draggers still pretending that "history" has a silent-H.

That's an accent thing. In some places the "h" is silent, and in other places it's not.

Is there any English dialect in which it's silent in "a history"?

That’s why the pronunciation of “an history” is such a contentious issue in British political linguistics to this day. Crucially, as has been observed correctly, the rule is that one should use the appropriate indefinite article. That in turn depends on whether one pronounces, as pirates, or has a silent ‘h’, revealing one’s origins even once the guise of Received Pronunciation had been defensively adopted.

Don't worry too much: I am a native speaker and I was taught the same in elementary school!