I'd ask the opposite: why would you use a vector index without the related metadata for filtering and hybrid querying. People don't use B-Tree indexes as a standalone separate database off to the side of their database. So why should they for vectors?