News for hackers. Stuff that matters.

A comment on Ant Group – scaling to 1.37M QPS on Ray.

Re:Ant Group – scaling to 1.37M QPS on Ray by george_123 on Tuesday December 13, 2022 @02:53PM (#33974755)
This is a guest engineering blog post from Ray contributors at Ant Group, discussing how Ant Group implemented scalable Ray Serving architecture atop Ray, deploying 240,000 cores for model serving, scaling by 3.5x from previous year, and reaching 1.37 million TPS during peak times.

(resubmission, URL redirected to /Engineering last time)

[ Parent | on HN ]

There are no comments on this comment yet.