Git isn't really all that fast for what it is. The "git is fast" thing came about as a comparison to non-distributed version control systems, where changing what branch you were working on could involve coordinating with some (often overburdened) server on the other side of the planet.

That's not to say git is slow of course. It's just.. basically what you'd expect given it's architecture.