You don't even really have to be that many people on a project for some limitations to become apparent.

With TFS and SVN, it's already enough to have stable versions you need to backport fixes and the occasional feature to in order to end up in branching hell, for example.