People often use a vector of vectors as a matrix. It's not as efficient, it usually is a bad idea, yet it doesn't discredit std::vector.

I suppose we could create a variadic template std::optional<...> which works like a tuple to solve your padding concerns.