Now that I think about it, don't most decent screens have multiple input ports nowadays?

For speakers you can just use a Y connector. Add a wireless mouse with multiple outputs built-in (eg Logitech MX Master), or a separate gaming mouse if you prefer, and that just leaves the keyboard to switch between the two devices.

Yeah, I've been doing exactly what you describe and it gets the job done. I just daydream of being able to press a button and all my peripherals are switched without significant lag.

Changing the input on my monitor requires three button presses, and comes with a noticeable lag. Then I've got to press a button on the bottom of my mouse, and another button for my keyboard. If I bought a fancy KVM, I'd be able to swap everything in a couple seconds! It'd be nice also to hook up a mic and webcam but I'll probably just think about it for another 6 months and never pull the trigger.