Re:Flashlight: Fast and flexible machine learning in C++
by v8dev123 on Friday April 16, 2021 @10:43PM
(#26841201)
You will see Modern C++ more like Python these days. It has really neat stuff. A comment is not enough to describe this.
Re:Flashlight: Fast and flexible machine learning in C++
by beltsazar on Saturday April 17, 2021 @12:37AM
(#26841616)
> There is no performance gain in going from Python to C++
The backend of TF/PyTorch is written in C++ anyway, so the more complex the model, the less time it needs to spend in the glue code (frontend) that is written in Python. Therefore, rewriting complex models in full C++, for example by using TF/PyTorch C++ API, probably won't much improve the performance.
In this paper the author rewrites some ML models in Rust using tch-rs (Rust binding for PyTorch C++ API) and finds the performance not that much better (even some models perform worse):
- Python allows for higher level description of algorithms, which means researchers can focus more on the ML stuff and less on low level details.
- There is no performance gain in going from Python to C++, because in both cases the models are compiled to specific binary formats to be executed on dedicated hardwares. TensorFlow enables accelerators not only for training, but also for data transformations and preprocessing.