At that point they've often turned into a type of drinking vinegar which, believe it or not, often don't taste acidic at all.
This is not true: if a wine turns into any sort of "vinegar" then a bacteria was present that should not have been. This is considered a "fault", and won't happen normally, even after 100 years in the bottle.
Hmm. I remember tasting a nicer higher end balsamic vinegar and you're right it didn't taste tannic or acidic. But that wine did taste different, in other words it still tasted like wine more than good balsamic vinegar.
A relative of mine did some work for a winery. As part of the payment or as a gift for a job well done they got a bottle of wine from 1965. The bottle was covered and dust and grime, looked pretty gross. By that time we drank it it had aged for about 25-30 years. The taste though was amazing. Didn't even taste like wine, had a very smooth velvet-y taste. Almost no acidity, not tannic at all.