One of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application.
Or even train in C++ or Rust without much loss in functionality.
Rust really has not had any presence in AI training engine yet, it's probably 100% c++.
I was referring to the libtorch library, which you can use through the tch crate. It is possible to make such rich bindings because so much of Torch is exposed through the C++ API. When more new functionality is moved to Python, it makes it harder to use functionality from the C++ interface and downstream bindings.
Facebook did similar thing to its original code PHP, it uses HHVM to 'compile' PHP(now called Hacklang) to gain performance, it seems doing similar thing with python here.
If I remember correctly, the JIT version of the PHP compiler proved itself against the AOT compilation to native code via translation to C++, so they went with the more productive variant.
The `torch.compile` API itself will not be available from C++. That means that you won't get the pytorch 2.0 performance gains if you use it via C++ API.
There's no plan to deprecate the existing C++ API, it should keep working as it is. However, a common theme of all the changes is implementing more of pytorch in python (explicitly the goal of primtorch), so if this plan works it could happen in the long run.
I'll admit I don't know enough about PyTorch to know what torch.compile is exactly. But does this means some features of PyTorch will no longer be available in the core C++ library? One of the nice things about PyTorch had been that you could do your training in Python then deploy with a pure C++ application.