Your article raises several valid points but is a overly optimistic. I still don’t believe that you AD through arbitrary code to produce useful gradient estimates; I’ve seen that fail miserably time after time.

I don’t think Julia’s generic code is the end game for fast code though. Efforts in Python toward code gen include projects like loopy, which targets accelerators, are still developing but make it easy to decouple kernels from loop domains, target specific devices and optimizations etc.