Actually, it could be an advantage to have people around who have only an average interest in your product - they can provide valuable insight into how the average consumer might react to your product. If your food startup is unconsciously geared toward the top 1% of picky eaters because everyone in your company belongs to that group, you're not going to have a very big market. If everyone in your car company drives a Ferrari, you're going to be at a disadvantage selling minivans to suburban parents.
It is always better to have at least an above-average knowledge of any industry you want to be part of. The narrative of outside players coming in and having success is very nice, but it's based on the premise that the people involved were originally very passionate and picked up the ropes very quickly.
Run a food startup? I expect the people there to be at least much more interested in the space than average, even if they're working on a purely technical role.
Music business? Events? Cars? Clothing?
Look, if you're going to be busting ass while getting funding and traction and you'll be inevitably spending a lot of time getting these things right, the kind of person involved has to have a personal interest beyond their job description. You need to understand customers and the business, and empathizing and knowing where to look in those spaces is more than just having business acumen.
You can hire without regarding culture fit if you're a medium-sized business and above where people have clearly defined roles and specialties. But when passion and interest are just as important as putting in the effort, I want to work with people whom I have more than just a cordial time at the office.