Wrong. Julia can call out to Cuda kernels, but it also does codegen for custom ops and types. Technically with enough engineering time, someone could rewrite all those C cuda kernels in julia, and julia would just generate low level machine GPU code on the fly.

Pytorch on the other hand HAS to glue together c and c++ cuda code.

You mean like PyCuda?

Nope. PyCuda is writing inline C cuda and hardly integrated with python.

Julia has four layers of GPU abstractions, each being successively easier to use ( bit a bit less control. ), all in pure Julia and reusing the compiler and normal Julia abstractions and integrated with AD etc

CUDA.JL, GPUArrays.jl, kernelabstracions.jl, tullio.jl.

At the highest levels you just write normal loops or index notation and the code and backward pass generated. Also generic to backend-simd and multithreading cpu and soon AMD rock, intel accelerator and distributed

This all composed with deep learning libs etc