What i meant when mentioning input devices is that the Pi would have it's own cursor, and I would have needed to handle the forwarding of inputs from the laptop's X server to the Pi's. This would have been a lot more difficult as I would have had to forward the inputs to the Pi only when moving the cursor out to the Pi-controlled monitor.

Keep in mind that I'm writing this from the top of my head, without googling anything or experimenting with the setup we're talking about. It's totally possible that I'm wrong.

Anyway, the hard part in this project was optimizing everything. The first proof of concept stream was pretty straightforward to get. Getting a slow PoC of what you described should be kind of easy as well. I encourage you to experiment yourself with the setup you suggested, you'll learn a lot more than you could possibly learn by reading a writeup. If you ever try it out, I'd love to hear about how it went.