Live in London in Zone 2 and there are absolutely tons of conker trees around me including in areas which are not posh. They are very common in an urban setting.[1]

[1] Which kind of sucks for me personally because they cause me really terrible hayfever. I think I'm specifically alergic to their pollen maybe.