Your last point: I would say that is generally true for all native-level speakers of tonal languages. It is interesting to watch them try to learn a different tonal language. It suddenly opens all these doors into their own language. "Oh, so that's why foreigners struggle with this sound." Mandarin: zai, sai, cai

When you read about tonal languages online, there is so much emphasis on "tonal languages are special". I don't understand why accent and pitch isn't added to the same bucket. In Japanese, accent and pitch is rarely taught, but incredibly important in daily life. Japanese is stuffed full of homophones which makes listening a tricky matter. BTW -- I am sure there are other "non-tonal" languages that I don't know about where pitch and accent are important (Korean?).

If you consistently pronounce words with the wrong accent or pitch, (average, uptight) Japanese listeners will refuse to understand it. Say what you like about it (fake/real/whatever), it is a common response.

It even matters in Latin languages somewhat. Which syllable is stressed can change meaning, sometimes avoiding ambiguity.

Exactly. I use the example of different tones when saying “really.” It can have very different connotations depending on the tone from questioning to sarcastic.

Mandarin, Cantonese, etc. have per-syllable tones. English, German, French and probably many more, have per-partial-sentence tones.