Exactly. For me, "going in to town" means going in to Croydon - a city centre with it's own large high street and multiple shopping centres at the outskirts of London. Croydon town centre again is at the centre of a collection of smaller villages within the borough, each with their own high streets.

I work near Waterloo, so I commute in to the centre, but outside of work I rarely see central London - it's pretty much a different city from my perspective.

I also found the mention of City curious. City is also pretty much a thing by itself. I guess it differs by what skillsets you put on your CV, but I've never worked with someone who has worked in City. I've only known one person who did. I don't come across recruiters trying to get me to take bank jobs very often. London is not defined by City any more than San Francisco is defined by its financial district.