Which happens exactly how often? I've been writing java code for the better part of two decades and basically never needed to swap my list/set/map types around. Also, one of the purported benefits of java's typing is your editor can find and refactor / swap all usages, so it doesn't seem to me like this distinction has much value. It instead makes me think it's a fetish for object orientedness...

In fact, the only cases I've ever swapped even a HashMap has been to trove, which implements data structures for builtin types rather than reference types. In which case using interfaces everywhere doesn't help.