Not confused, just not talking about PCs in general (although I have heard cases of this happening there but that might be just indeed demo groups); I still do this on those old computers today for hobby purposes (the last work I did was somewhere end 90s on a legacy system which counted on video memory and which had to be ported so I had to remove & test that 'feature') ; there just isn't another way to get memory if you don't want to cheat (and solder in 1024kb).
Edit: I don't know if it was used in practice because there were so many different VGA cards, but [1] shows that it was common to have 256kb on a card which, depending on the era, was significant. And like said above, for business applications in text based environments (screen 0) you can use a large portion of video memory for storage while you still see something. But I can see the issue there; with other computers we didn't have that issue as you could trust a certain amount to be there for sure.
"Extended memory" drivers were a common thing back then but that was to circumvent the 640KB/1MB limit of DOS and they had nothing to do with video card memory. In that era few people even knew how much memory was on their video card because it was rarely a concern, most of the time it was just barely enough to hold a single frame buffer, typical of most VGA and "Super-VGA" cards.
Extended memory is that which is beyond the boundaries of the original IBM PC design: https://en.wikipedia.org/wiki/Extended_memory
It's not video card memory in any way. If I recall correctly the video card memory was often mapped into the 640KB-1024KB memory zone, but using that for system memory was futile since you'd be unable to see anything on screen.