I disagree with the idea that "your VP of marketing should be an engineer".
An engineer may be able to test before or after the fact but a real marketer already has a framework to start from that gets you on the dartboard.
All things equal sure it would be nice to have a marketing person who is creative but also an engineer. But the world is not perfect. And the best brain surgeon is rarely also going to specialize in performing heart operations.
I think you are also discounting what I will call "get a law degree" effect. Sometimes the training than one has (or how your brain thinks that even allows you to pursue some profession) is contraindicated in another profession. With a lawyer this would be the tendency once trained like a lawyer to always think of problems as a lawyer would (and not be able to take appropriate chances that a non lawyer might).
Engineers can be creative of course and they could be marketers. But as a generality I don't think the benefits of thinking everything has to be tested and optimized is so clear cut.
Saying the VP of Marketing should be an engineer is like saying the VP of Engineering should be a marketer.
I think we want to have leaders that scale their functions, are aware of their personal shortcomings, and hire for those. In the case of a VP of Marketing who is not an engineer and in which there is a need for technical investments in the marketing function, let's hope that that person can hire a great technical lead.
It would be the same if we had a VP of engineering with no business foundation. I would hope that person would hire a great product management staff or that he leans on whoever is the business thinker in the company.
I think in general we would hope companies hire for diversity of skills at the time those skills are needed depending on the lifecycle of the company.
I always appreciate an engineer who can think in terms if users and business and not just how to for(;p++=q++;); just to impress his or her friends.
What world are you talking about? The startup world? There is a big big world of marketing that exists separate from startups, YC, and venture invested companies.
Edit: AKA "the echo chamber"
The fastest companies I know today have a ton of engineers dedicated to their marketing efforts. It's not enough to have the best product. You have to be able to find and acquire customers in a affordable and repeatable way. A non-technical marketer will say "I was thinking about it all night, and I like this wording the best". A technical marketer will say, "I tested 100 variations and these 3 performed the best. Let's try and optimize them even more." That's not to say there isn't a place for creativity and intuitive decision making, but a good marketer will always be seeking data to validate their hypothesis.
If you don't have a technical VP of marketing, the engineers should report to the CTO and should treat the marketing team as their customer.