Yes precisely - it's not the frontend that's doing the lifting here. The interactivity as you describe it depends on if the backend supports it, which not all do.

I agree that there's room for improvement here, but I also think that it won't happen in Plots.jl (and if it does, it'll depend on a backend that already provides interactivity). I think it'll be more likely to see something like this plop out of Makie.jl or something built on top of Makie.jl, as that has a lot of primitves for interactivity available already.