Agreed, the biggest issue is durability, like you I've had many broken zippers which often seem to break at the most inconvenient time, embarrassing experiences such as having to secure my fly with bent paperclips pushed through the fabric come to mind.

But I think there's more to it than just durability as there are some basic design issues that are hard to overcome. For instance, if a tooth gets pulled out or it does not mesh properly with its opposite mate then all will unravel (and that happens surprising often). It only takes one missing tooth or the first teeth to be not properly anchored for the whole zipper to fail. Cascading failures don't happen with buttons (one missing isn't much of a problem).