I agree. Low or lack of mobile visitors was the only good reason for not using responsive design. Regarding time, it's all experience. Personally I don't think it takes too much extra effort to implement. Constraining the design is more of knowing how to design responsively. I'd much rather read an optimized representation of the site's content than having to zoom in and out of the "static" site. I think the question is more "do we want to support mobile at all?" If you do, then support it!