It might not be a requirement to memorize all of the words, but then again it's hard to tell which ones will still be important later on. It does add a significant mental load not only trying to pronounce the words, but try to remember them and then try to recall those memories when the word is referenced later in the story.

I had a hard time with Game of Thrones for the first few seasons too, because of the absolutely massive cast of characters. It became easier when most of them died off. It's one of the reasons I don't read a whole lot of fantasy novels, every author considers themselves a Tolkien and has to show off how much work they put into world-building.