Yes in that specific case, perhaps the candidate was spooked by the speed of the offer, but next time might be the opposite: another candidate will get tired of the lengthy process until you get to the point and will drop out in favor of a company looking for more than ticking all boxes, especially as it might be a red flag that the environment isn't flexible enough to adapt.

So don't feel bad for following your guts and making an offer: if he really had been the perfect fit, he would have felt the same way.