It's going to kill morale if you keep that guy around. If they're talking to you that much about him, there's probably a lot more they're talking about that you never hear.

Fit goes further than skill, in my opinion. If you want to keep your non-tech folks around, let that guy go, and hire for fit.

You'd be doing yourself massive favors.