Show HN: Panini AI – A platform to serve ML/DL models at low latency

Re:Show HN: Panini AI – A platform to serve ML/DL models at ... by tedivm on Monday March 11, 2019 @07:19PM (#19363853)

Is it actually 3x faster, or does it just scale more?

In other words, if I had a model that previously took three seconds to get a response from would this platform respond in one second?

[ Parent | on HN ]

Re:Show HN: Panini AI – A platform to serve ML/DL models at ... by avin_regmi on Monday March 11, 2019 @09:01PM (#19364451)

Sorry, I should've been more clear. Both predictions for TF serving and panini serving was done in a single thread in the same specification machine. We used a simple model for image classification of CIFAR dataset. Roughly, 500 predictions were made for panini and 200 predictions for TF serving. The graph on the website is for throughput. I'm planning to write a medium post soon regarding the benchmark test.

[ Parent | on HN ]

Re:Show HN: Panini AI – A platform to serve ML/DL models at ... by dang on Monday March 11, 2019 @07:20PM (#19363861)

We changed the title from "Show HN: 3x Faster Than Tensorflow Serving" to what the page says, which is less baity.

https://news.ycombinator.com/newsguidelines.html

[ Parent | on HN ]

Re:Show HN: Panini AI – A platform to serve ML/DL models at ... by tedivm on Monday March 11, 2019 @07:33PM (#19363953)

That's definitely an improvement, but I'm hoping someone from the Panini team will step in and clarify regardless.

[ Parent | on HN ]