News for hackers. Stuff that matters.
Re:We improved Tensorflow Serving performance by over 70% by ajtulloch on Monday March 25, 2019 @11:00PM (#19488927)
FWIW, I believe that the current state of the art for batch-size 1, fp32 inference for ResNet-50 on Intel CPUs is AWS's work in https://arxiv.org/abs/1809.02697. After the low-hanging fruit outside of model execution are picked, this kind of work is probably quite relevant.
Re:We improved Tensorflow Serving performance by over 70% by human_afterall on Monday March 25, 2019 @11:21PM (#19489019)
Hey! Author here, thanks for linking the paper. The article was from an infrastructure perspective, but we're definitely diving deeper into graph execution optimizations after this:)
Re:We improved Tensorflow Serving performance by over 70% by rahulun on Monday March 25, 2019 @11:30PM (#19489074)
Are there any particular optimizations you are looking into ?