Scanning the comments, it seems broken that there's no way for an end user to clear GPU memory. Particularly a way that works across platforms, or at least across drivers would be ideal.

If it can persist across restarts... does a shutdown do anything differently? I'm not too familiar with GPU programming, but there has to be something we can do.