Stabbot is open source. It's written in Python and calls out to ffmpeg with vidstabdetect/transform: https://gitlab.com/juergens/stabbot

I don't think vidstab can do exactly what's being asked for here, I've never seen it rotating the frame completely to keep a rotating pen stabilized, or move the frame inside a larger canvas to keep a moving subject centered. I think you have to use video editing and do that manually.