You’re right let’s call them muggles, which has the added double-elitism of “I understood that reference” as well as the in-story elitism of the wizards. /s