I've worked in a team of 4 (and later, of 5), and it was amazing: everything could still be communicated quickly, but you didn't feel overwhelmed if a team member was inaccessible.