It gets in the way of your programming immediately - the friction involved in bootstrapping a viable emacs environment that unlocks all the potential everyone is always going on about, is very high.

Deep diving into emacs is not a productivity booster - its a hobby that one does for its own sake, if that's the kind of thing that floats your boat.