For many domains, it's a big worry. I routinely encounter large, sparse datasets where I/O and storage dominate other concerns. Big sparse tensors are worse than the "slowness" associated with a suboptimal sparse vector math implementation.

"I think this is one of those shibboleths that separate folks who are experts in ML/data-science at scale vs those who are coming more from a enterprise/DBA background."

I've been doing ML work for a long time. This is actually a shibboleth that separates folks who mostly do image classification from other kinds of ML people.