Dear god yes. If everybody on your team just has a generic math/CS background and no one has a deep understanding of the domain you are working in, then you'll end up wasting so much time you might as well give up and go home.