That problem doesnt exist for me using Win8.1 on laptop and different screens at home and work. Make sure to always use the Win+P "Extend" option, and when first used, select to set the external as primary. Then should work ok. i lost hours to figure this out....same situation as you with need to take my gear to meetings.
So then you might decide to use just the external monitors, and scale it back to 100%. That works pretty well, but Windows forces you to restart (or "Sign Out" and back in, which will close all of your programs and stuff you were working on) every time you change that scaling option. Great! Everything looks fine...until you disconnect the Surface from its dock. Now it's back to looking tiny, and to make use of the SP4 internal screen I've got to once again change the scaling, restart, etc.
People at work ask me why I'm always swearing whenever I come into board meetings. It's because I've just detached my SP4 to bring it into the meeting, and I've either got to drop everything I'm doing to restart and be able to use it during the meeting, or keep the 100% scaling to avoid restarting and squint at my SP4's screen during the meeting.