I’m assuming old multi story houses with gas heating typically use a water system (radiators) and only a single basement furnace? In that situation, switching the boiler to anything else or even better just using central heating should be a reasonably sized investment.
It’s not like German cities are unique in how they are built and heated. Lots of other cities have done this conversion in the latest decades.
Gas infrastructure was old and established in all major European cities. Now it’s electrical stoves and central heating in many places.
What's "central heating"? I believe Germany already uses central heating in most places. The question is what powers it. Can it be efficiently replaced by renewable-energy (!) electric heating vs. fossil? Other commenters have posited "no".
The way I see it, investing in nuclear should have been part of the plan, instead of replacing it with renewables (vs. doing so for those uses of fossil where renewables make sense). The fear of nuclear energy in Germany is just ridiculous IMO, and arguably has become a self-fulfilling prophecy. With nuclear power on its way out know how and effort to research and to maintain existing plants and investment in it are at a minimum making continued operation now a liability.
Centrally burned fossil is better for the environment than burning in residential buildings, especially in cities. For the climate it makes no difference though.
Even though it’s a waste of high grade energy, one could imagine powering heating plants with electricity in a pinch. If there is so much surplus electricity that reactors are closed then perhaps…
The Stockholm City gas system, although much smaller than Berlin’s was swapped to a renewable gas (biogas) 10 years ago.