That's what we have done (algo trading). Our research backend uses pandas, but we ended up taking about a month removing it from prod code. It does surprising things with memory usage, and the functionality we needed was more or less wrappers around numpy anyway. Most of our performance critical code is in cython as well. For this trading application, speed obviously isn't the biggest concern, so python+numpy is fine. It is C++/Java everywhere else though.