I know at least a couple places where you can buy it (Huis Maria in Harelbeke, Vanuxeem in Ploegsteert, so both Flanders and Wallonia) so I would assume those to be widespread enough throughout the country.
To me coming from France it was Belgium that was the easier place to find glass-bottled water :)
The bottles are reused, and the water tastes a lot better than at least my tap water (in Antwerp - the tap water comes from a stagnant canal used for merchandise shipping, and last year for example it turned green and smelled of algae for a while after a ship carrying fertilizer capsized).
And yeah, you usually give the bottles back. However, it's much more common in Germany than in Belgium, it seems. I have never seen a line to give the bottles back in Gent, unlike Berlin.
(a) I used to live in Malaysia where tap water wasn't drinkable as-is, but filtering and boiling made it drinkable and tasted just fine. Hot water dispensers are pretty standard to have in most Asian homes; just put the water through a filtering pitcher before pouring into the hot water dispenser.
(b) Subscribe to those 5-gallon big blue bottles and a dispenser. They get actually reused instead of downcycled.
Why is your water quality bad?
You can get distilled water delivered in very large glass jugs (4 gallons i think?)
https://www.who.int/water_sanitation_health/dwq/nutrientscha...
This is mostly about dissolved minerals, carbonates etc, and you should be able to control trivially.
I know I would prefer to know exactly what's in my water.