It's more than performance. Some of the worst code I've worked on has had too many layers of sub-classes, making it difficult to navigate and a real loss to developer productivity. After a certain point, it becomes OO spaghetti or, more accurately, "lasagna." At more than 3 layers, you really need to stop and think if it's necessary.