Halide and OpenCV are not comparable. Halide is mainly about decoupling the description of a low-level image processing operation from implementation details regarding tiling, storage of intermediate results, etc. OpenCV is a library implementing high level algorithms like multi-view 3D reconstruction that Halide does not even attempt to address.

My mistake, I phrased my question incorrectly. Why not opencv with a halide backend. The tensorflow whitepaper compares the TF ir to the halide ir. In fact, the work in tensorflow core is around XLA https://www.tensorflow.org/performance/xla/ (which brings compatibility across various architecture)

It seems that this is more an attempt at locking the tensors to target custom metal, so that it can't be copied and made to work on other phones.

For examples, the Google Camera NX version is a port of the HDR+ feature on compatible phones.. Even the cheap Xiaomi. They actually give results comparable to Nexus.

However, by locking the model to a very custom image processor (and custom IR) , this can't be ported. This looks to me as a version of walled garden more than a performance win.

Halide is much more flexible. OpenCV is a fixed high level toolbox, combining things requires lots of intermediate buffers. Halide allows compiling arbitrary kernels in a fused way with extremely good performance. Sure you could implement OpenCV functions using Halide, but better to expose the lower level more flexible API. Source: I’ve rewritten parts of an OpenCV app in Halide for performance.

Also your argument that it’s not about perf makes no sense to me. Sure other phones have HDR, I bet its way slower and more energy intensive than the Pixel 2. Except maybe the iPhone, which may also have custom silicon. If they had custom HDR tech in software they wanted to keep they could just keep the source closed, or patent it.

That bet is the problem. Tensorflow on mobile already targets GPU and is going to be even more so with XLA. This debate is going to be hearsay unless we have an apples to apples comparison of the base tensorflow code running on the halide ir targeted to the tpu vs a vanilla ARM GPU... Even using opencv.

I find it telling that they compared cpu vs tpu... Not GPU vs tpu.

I think you’re confusing two separate things. There’s two separate approaches to a computer vision task: classical and deep learning. Halide and OpenCV are for classical approaches, TensorFlow is for deep learning.

People don’t use both for the same problem, nobody is using “tensorflow code running on the halide ir”.

Both approaches use a lot of basic operations applied to matrices, so the custom ASIC supports either, but separately. HDR sounds like a better fit for a classical approach, so their HDR filter is probably written in Halide, with no machine learning or TensorFlow anywhere.

I'm really confused why you keep bringing up opencv. What does opencv have to do with the (presumably) TF ML model running on the IPU that you want to benchmark against the TF ML model running on a GPU?

Because halide is positioned as a high performance improvement over opencv. While I believe it is just an architectural decision to enable a non-portable ML model.

But Halide does have higher performance, is portable to many platforms as well as GPUs, CPUs (w/ parallel SIMD), and isn’t used for ML models...

Where did I mention halide?

My question is why do you care about opencv when benchmarking neural network performance on various hardware?

Curious if anyone here knows anything about this alternative for existing phones:

Ask HN: What happened to tensorflow lite? | https://news.ycombinator.com/item?id=15494368 (Oct 2017, no comments)

>zitterbewegung: They announced it at Google I/O in May See : https://techcrunch.com/2017/05/17/googles-tensorflow-lite-br... I can't find another announcement is this vaporware?

It became tensorflow slim: https://github.com/tensorflow/tensorflow/tree/master/tensorf...