Agree on the size not being the issue. I transitioned from a data engineering manager to a data product manager at a new company. You know how much data a typical customer generates? Less than 1TB a year.

I told my VP that the engineering foul ups in the current product are easily fixable. Standard tooling and patterns exist to re-architect and solve the bottlenecks. What is much harder is a data architect to make sense of the complex data and make sure there is good value for our customers.

Guess what position I don't have on the team, and won't have due to budget issues.