There are a number of components here which enable (what I would call) the expression of more advanced models using Julia's nice compositional properties.
Flux.jl is of course what most people would think of here (one of Julia's deep learning frameworks). But the reality behind Flux.jl is that it is just Julia code -- nothing too fancy.
There's ongoing work for AD in several directions -- including a Julia interface to Enzyme: https://github.com/wsmoses/Enzyme.jl
Also, a new AD system which Keno (who you'll see comment below or above) has been working on -- see Diffractor.jl on the JuliaCon schedule (for example).
Long story short -- there's quite a lot of work going on.
It may not seem like there is a "unified" package -- but that's because packages compose so well together in Julia, there's really no need for that.
The reason for the lag is that Julia has been focusing on general composable compiler, codegen and metaprogramming infrastructure which isn't domain specific, whereas pytorch and friends has been putting lots of dev money into c++ ML focused optimizers.
Once the new compiler stuff is in place, it would be relatively trivial to write such optimizations, in user space, in pure Julia. Then exceeding that would be fairly simple also, plus things like static analysis of array shapes
What we've noticed is the vast majority of the time it's the data scientist's code that's slow not the actual ML model bit. So allowing them to write very performant code with a dumpy-like syntax and not have to deal with painfully slow pandas, lack of true parallelism, etc. would be a true game changer for ML in industry.