How many millions of years ago would we need to go back, to find the common ancestor?

Whales and deer are not alike in the same way.

Same question, I suppose. A longer span of time, or shorter?