IMHO this is the problem that occurs when things are built by committee. You need a strong person who "owns" a creative project, lives and breathes it, and sees it through.. like a movie director or an author does. We shouldn't be thinking of Valve's motivation to make the game, but some creative who defines what it means to be "Half Life".

Valve's structure doesn't feel very supportive of this sort of creative game making even if it's perfect for running infrastructure and creating storyless games like Dota and TF.