You are absolutely right, peoples perception of Frankfurt is mostly wrong. But the size of Frankfurt largely contributes to the quality of life here. You can very easily traverse the city by bike, the public transportation is fast and quick, and the big banks attract lots of capital. By day Frankfurt grows to about 1.5 Million people who just come to the city to work here, so it feels much bigger while still having everything happen on a rather small scale.

The biggest downside would probably be that Frankfurt is at the top of nothing. We have very good theatres, very good restaurants, nice outdoor activities, but compared to other cities, you don't come here for any specific thing, you come here for the sum of things that make it a very attractive city to live in. The airport is reachable very fast from the city center, i was able to traverse between a Frankfurt Office and a London Office in about 1:20 door to door, so its geographical location makes it also very attractive when visiting or working somewhere else on the continent. As local novelties DECIX is located here and many large Data Centers have popped up all over the area, so if you are interested in Infrastructure there's a lot of jobs and know how in the city.

Frankfurt is very much embedded in its surroundings, with many smaller interconnected towns and several mid-sized cities within reach. [1] This gives access to up to 5.5 million people, 2.2 million in Frankfurt's close vicinity. Many of those happily change city for work, especially since the distances are quite short. Traffic is usually a worse problem for travel time than distance.

You also have the somewhat unfair situation of smaller towns right next to Frankfurt offering lower corporate taxes (part of which is levied by the cities) while taking advantage of Frankfurt's infrastructure (which is also paid with the corporate taxes paid in Frankfurt). This attracts a lot of businesses, of course, and on paper detracts from Frankfurt's size when all that separates them is one Autobahn.

[1] https://en.wikipedia.org/wiki/Frankfurt_Rhine-Main

Londons financial area is basically City ("the square mile") and Docklands - two tiny little parts of town, so if you're going to go by size it's meaningless to compare all of London.