There was still a bit of a split in the market back then between those who went TNT2 (amazing, at the time, image quality) and those who went to the Voodoo3 (good raw speed, but limited to 16-bit color and thus noticeable banding artifacts)... 3dfx still held on to a bit of a niche there for a while longer, but then once the GeForce and GeForce 2 came out 3dfx was just dead in the water with nothing that could compete anywhere on the performance versus image quality axes.
https://www.vogons.org/viewtopic.php?f=46&t=36548
Texture memory capacity, texture size limit (256x256!) and lack of s3tc/DXT1 compression were the real culprit. TNT shipped with 16MB, in TNT2 32MB came standard. Texture quality played the biggest role in perceived visual difference. Not to mention TNT performance went 1/2 down in 32bit mode, TNT2 fared only slightly better.
Quality is great, until you learn both Nvidia and ATI repeatedly cheated in benchmarks and games dropping quality below old 3dfx limits. ATI Quake 3 was quite dramatic: https://techreport.com/review/3089/how-ati-drivers-optimize-...
Geforce didn't come out until Oct 1999 and even then it was only one chipset. The rest of the product range came the following quarter (2000).
I skipped that generation and jumped to the Geforce 2 (this time first party rather than a 3rd party card with nvidia GPU), which was also an awesome card.
As for the fall of 3dfx, I dont think the rise in popularity of DirectX and OpenGL helped their case as that really levelled the playing field. After that, specifically targeting Voodoo cards felt more like a chore than an enhancement.
Wait, nVidia made their own cards? I can't ever recall seeing one that wasn't made by a partner.