The only issue I found was random disconnects when using a backlit LED keyboard and the stream deck on the same non-powered USB switch. I switched to a powered USB switch and that issue is gone.
The switch I'm using is IOgear 2x4 with a USB 5v 2a power brick. https://www.iogear.com/product/GUS402/
Or it does but the state of DDC/CI (and HDMI-CEC) is very sad as implementation is inconsistent and subpar to say the least (from order of commands to timing to subtle difference in required payloads that would make the display reject commands).
A dedicated device would cut the GPU out of the equation, but most likely patching ddc tools can be equally fruitful.
In addition to the other links posted, ddcutil.org has some more good info: https://www.ddcutil.com/#introduction
My use case is a bit different. I have two Windows machines and a dual monitor setup. I use NirSoft ControlMyMonitor which has a CLI and a Elgato Stream Deck.
In a normal KVM approach both monitors are showing only one computer. My use case is a bit odd- I want one computer on monitor A and the other on monitor B. I may also want to flip that. I use a portrait and landscape setup. Then I want to be able to switch mouse/keyboard independently of monitor configuration.
Each computer runs the Stream Deck software and the Stream Deck is connected to the USB switch.