LLM Inference Handbook

Re:LLM Inference Handbook by srameshc on Friday July 11, 2025 @11:13AM (#44533089)

If I remember, BentoML was about MLOps, I remember trying it about a year back. Did the company pivot ?

[ Parent | on HN ]

Re:LLM Inference Handbook by fsjayess on Friday July 11, 2025 @11:20AM (#44533173)

There is a big pie in the market around LLM serving. It make sense for a serving framework to extend into the space

[ Parent | on HN ]

Re:LLM Inference Handbook by aarnphm on Sunday July 13, 2025 @08:57AM (#44550121)

Hi srameshc, the core of BentoML is still considered MLOps. A lot of our customers are pretty much MLOps users. However, LLMOps seem like a natural progression of the product, given that a lot of our users now want to experiment/build with LLM-based services.

[ Parent | on HN ]