I see it as a kind of side-channel defense; take the set of passwords which it is possible to crack quickly and use the work factor to even it out so that the total time to crack a password remains relatively constant.
Of course it's a good point that the challenge then becomes determining what that set of passwords is. And not having run the numbers, I can't be sure how much room for improvement there is.
BCrypt is already based on the premise that there are passwords which are easy to compute, and tries to avoid that differentiation. Why would you introduce it back in?