Weird, I'm the exact same way. I can concentrate perfectly fine in a coffee shop, but at home it is difficult unless I have absolute silence and total solitude. The presence of one person (even if they're not bothering me) is enough to completely derail my ability to concentrate. I think it's because in the back of my mind, I know that person might interrupt me, and just the "threat" of interruption is enough to prevent focus.

I wish I was like von Neumann sometimes. He was known to do his best work on quantum mechanics in the living room with his whole family sitting around talking and the television blasting.