Yeah, I've met people like you. The problem is, this asks from your interlocutor of awareness for internal USA geography knowledge. On the other hand, many of those "from San Francisco" or from somewhere alike expect you to be just "from Europe" instead of "Germany" (or any other national state with its own cultural identity and external policy, for that matter).
P.S.: Try "Franconia" or "Thuringia" instead of "Germany" and see how many bells that rings! Mind you, these are broad regions, not merely cities.
I often say "London". I don't like saying "England", I consider the whole UK to by my country, but "United Kingdom" or even "Britain" are often not understood, and in any case, "which city?" is almost always the next question.
There are exceptions. If I'm introducing myself to someone with very limited English, like a young child in a developing country, I'll say England.
I wouldn't have the same expectation of somebody from Asia. Not only is language a possible problem, but there are a TON of large cities in China (and other parts of Asia) I know nothing about.
That city has been part of the American "brand" for decades; from hippies, free love, Vietnam protesters and gay rights, to the dot-com boom and more recent startups. Add some destructive earthquakes, plenty of American films and music, and the international press whenever a startup does something bad.
Some (many?) Americans would describe themselves as hailing from a specific state or city, not the nation.
Personally, when traveling abroad, if asked where I'm from, I generally say "Washington DC", not "the US".