I tend to use Anaconda where possible, makes such dependencies easier to manage (but still not trivial... though you can change cuda runtimes with Anaconda, the actual driver is a system-wide dependency, of course).