Getting a good home-office setup is critical here. In my previous house I had a separate office and enjoyed working from home. In my current house we do not have a separate room (adding it now) and I can't get anything done there (doesn't help that the kids are home from school).

On the response times, if your office is one of constant interruption for that, then you can simply just have video conferencing on all day rather than just for meetings. This helps with "Hey Bob, what's the answer to this?". It has the same level of distraction as being in the office and just interrupting each other all the time. (which is awful for some, and useful for others)