Re:Onnx Runtime: “Cross-Platform Accelerated Machine Learning”
by fisf on Tuesday July 25, 2023 @12:22PM
(#36864782)
Onnx is just a serialisation format (using protobuf iirc) for the network, weights, etc.
Your training framework and a suitable export is the compiler.
Onnx Runtime (which really has various backends), tensorrt, .. (whatever inference engine you are using) is your JVM.
Re:Onnx Runtime: “Cross-Platform Accelerated Machine Learning”
by nerpderp82 on Tuesday July 25, 2023 @01:55PM
(#36866516)
That is my understanding, ONNX is the weights and the operators. You could then project that model into SPIR-V, Verilog or run it via native code.
Re:Onnx Runtime: “Cross-Platform Accelerated Machine Learning”
by snnn on Wednesday July 26, 2023 @09:48AM
(#36877657)
It is true if your model is statically shaped. ONNX is also a collection of C++ code , a C++ library for doing shape inference. You cannot project the C++ code to another high level programming language.
Re:Onnx Runtime: “Cross-Platform Accelerated Machine Learning”
by nerpderp82 on Tuesday August 01, 2023 @12:01PM
(#36957833)
The model itself doesn't contain C++ code. The runtime supports Python, JS, etc. I don't think they are shipping a copy of clang as part of the rt. I could be wrong, I haven't looked. Of course it needs the scaffolding to get data in and out.
I just did an install of the runtime on Python ( pip install onnxruntime ) . Here are the additional packages it installs.
Package Version
------------- -------
coloredlogs 15.0.1
flatbuffers 23.5.26
humanfriendly 10.0
mpmath 1.3.0
numpy 1.25.2
onnxruntime 1.15.1
packaging 23.1
protobuf 4.23.4
sympy 1.12
Re:Onnx Runtime: “Cross-Platform Accelerated Machine Learning”
by snnn on Saturday August 05, 2023 @12:58AM
(#37009150)
ONNX is a spec. ONNX Runtime is an implementation of ONNX. There are other implementations too. But ONNX is not a text spec like the RFCs for network protocols. ONNX is also a collection of C/C++ code. ONNX's implementations rely on this code to do type and shape inference. My point was: if someone wants to implement ONNX(write a library that can load and run ONNX models), he/she has to reuse this C/C++ code, or totally rewrite them in his/her favorite programming language(but I think it is not very practical).
If an ONNX implementation wants to do codegen, like what XLA does, then usually it is based on LLVM and it needs to be shipped with a copy of LLVM.
Is it? From what I understand, to use an analogy, ONNX is the bytecode specification and JVM whereas Pytorch, TF and other frameworks combined with converting tools are the Java compilers.