Multi GPU in PyTorch is usually just a few lines of code. Can’t be any easier. Wrap model in DataParallel and enjoy life. Last time I touched TF, things were pretty gnarly for this frequent use case. Don’t know how it is now.