I did that as well for some time. Consider not taking a whole day off, but but taking off two hours here and two hours there. This way you seem much more available but may actually be even more flexible in how to spend your free time. Of course that requires a lot more flexibility on your part. But the payout in time and career success is worth it.