There's another unexpected bonus to nvidia-docker; CNTK leaks memory if you abort training prematurely, causing the GPU to go OOM. But since the memory is owned by the nvidia-docker process, killing-and-restarting the container fixes the issue without having to restart the entire system.
There's another unexpected bonus to nvidia-docker; CNTK leaks memory if you abort training prematurely, causing the GPU to go OOM. But since the memory is owned by the nvidia-docker process, killing-and-restarting the container fixes the issue without having to restart the entire system.