LLM Microserving: a new RISC-style approach to design LLM serving API

Posted by jinhongyii on Tuesday January 07, 2025 @02:58PM
from the blog.mlc.ai dept.

( 1 comment | 4 points | on HN )

Re:LLM Microserving: a new RISC-style approach to design LLM... by crowwork on Tuesday January 07, 2025 @03:02PM (#42626725)

Scale LLM serving with programmable cross-engine serving patterns, all in a few lines of Python

[ Parent | on HN ]