Peace and quiet is so that you, candidate, can focus on the answers. I personally don't care if your kids organize the party in the same room, but I care if it causes you, candidate, to not be able to fully focus on the interview and then me, the interviewer, to have trouble understanding whether your behavior was due to disturbance.
Interviewing is an expensive investment. I spend a lot of time doing the interview and I want to use what little time we have to the fullest extent.
I think I’m ok, when some life trickles through, while I’m interviewing others. With people not always having the comfort of a separate room to hide in. A remote setup with 2 toddlers running around is sometimes a necessity. And I wouldn’t want to reject someone solely on that basis.