It's also not about accent. There are distinct Sephardic reading styles. It's not technically doctrinally different either, at least formally. In religious terms, distinctions are termed is "customary/minhagim" which are lower on the hierarchy.
In any case, etymology is not far off the mark. Ashkenazi judaism isn't just named after germany, it originated in the HRE and Ashkenazim spoke a German dialect.
My grandmother was a native polish speaker, secular, and would not have identified as "askenazi" before the war. She identified as polish, strongly, and was as comfortable in a sephardic synagogue as an ashkenazi one. My grandfather, a Yiddish speaker, was more comfortable in an ashkenazi synagogue. Most are mixed, these days, whatever the majority is.
Naturally under the Ashkenazi umbrella there were people of different cultures (and different degrees of assimilation to the surrounding non-Jewish population), but those Jews were still more similar to one another than to non-Ashkenazi Jews.