We improved Tensorflow Serving performance by over 70%

Re:We improved Tensorflow Serving performance by over 70% by ss7pro on Tuesday March 26, 2019 @02:08AM (#19489607)

Have a look here: https://github.com/IntelAI/OpenVINO-model-server/blob/master... You can replace tf-serving with OpenVINO to get even better performance and latency when running on CPU

[ Parent | on HN ]

Re:We improved Tensorflow Serving performance by over 70% by londons_explore on Tuesday March 26, 2019 @02:44AM (#19489724)

What useful models run at decent speed on a CPU these days?

Even basic image classifiers tend to be 100x faster on a GPU or TPU...

[ Parent | on HN ]

Re:We improved Tensorflow Serving performance by over 70% by bitL on Tuesday March 26, 2019 @04:34AM (#19490081)

Inference is not that super slow on CPU, especially for network requests that already have quite a bit of latency, so plenty of companies use CPUs on the cloud for lambda/flexible loads where GPUs aren't available.

[ Parent | on HN ]