The following article goes into the technical details of this phenomenon. TLDR: it's harmless, caused by an unnatural increase in volume as you move across the frequency spectrum (from great noise cancellation at low frequencies to nearly none at mid-frequencies).

https://www.soundstagesolo.com/index.php/features/178-eardru...

Earbuds don't tend to exhibit it nearly as much as headphones.