Re:Snips Uses Rust to Build an Embedded Voice Assistant
by throwawayendy on Thursday February 22, 2018 @10:24AM
(#16438127)
And also removing all of the overhead of the machine learning abstraction. Although I am skeptical that anyone would ever actually do that
[ Parent
| on HN ]
Re:Snips Uses Rust to Build an Embedded Voice Assistant
by danieldk on Thursday February 22, 2018 @02:14PM
(#16440463)
I used to train my parser (before switching to Tensorflow) using Caffe, dumped the parameters using a small program, and loaded them up in Go arrays slices and applied the network using simple C BLAS operations. This works fine, especially when you are using simpler networks. As a bonus, you don't have the overhead of Tensorflow session runs.
It does become a bit of a drag when you are building more complex networks (e.g. with multiple RNN layers, batch normalization, etc.). In that case there are two straightforward options for Rust. There is a Tensorflow Rust binding against the tensorflow C API [1] with which you can just load and run a frozen Tensorflow graph. This is the approach that I am currently using, though I am running graphs on workstations/servers.
Another option is compiling the graph with Tensorflow's XLA AOT compilation, which compiles the network to C++ classes (that you could bind from Rust).
Re:Snips Uses Rust to Build an Embedded Voice Assistant
by albi_lander on Friday February 23, 2018 @05:09AM
(#16445373)
There seems to be a third option provided by someone, kali, working at Snips: https://github.com/kali/tensorflow-deploy-rust
Re:Snips Uses Rust to Build an Embedded Voice Assistant
by danieldk on Saturday February 24, 2018 @01:23AM
(#16452039)
This only supports a small subset of ops. It is pretty much corresponds to the first option that I mentioned - train with Tensorflow, extract the parameters and provide implemenations of ops in your native language.
Re:Snips Uses Rust to Build an Embedded Voice Assistant
by quadrature on Thursday February 22, 2018 @10:46AM
(#16438320)
Not exactly sure what you mean. After the model is trained there isn't much "machine learning abstraction".
you can serve the model using tensorflow serving using exported parameters. Its exactly what you want for serving the model.