Ask HN: What do you use for ML Hosting

Re:Ask HN: What do you use for ML Hosting by thundergolfer on Tuesday May 02, 2023 @05:25PM (#35794435)

> are worth avoiding so you don't get stuck with somebody else's framework.

Modal eng here. Modal is not setup as a framework. Think of more as Python-defined serverless infrastructure that has native support for the Python runtime. This is in some places called "Infrastructure from code", as opposed to "Infrastructure as code" which means just source-controlling K8s YAML and Cloudformation.

A major benefit of this approach is that the cloud becomes part of your dev loop, as opposed to doing `docker build`, `docker push`, `kubectl`, etc just to ship a change to a GPU.

In the script I posted Modal APIs are mixed in with standard Python code for brevity, but many customers just keep their code in their own modules and have a `modal_infra.py` module that defines the serverless infrastructure.

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by version_five on Tuesday May 02, 2023 @05:29PM (#35794479)

Understood, thanks for clarifying. I'll edit my post.

[ Parent | on HN ]

Re:Ask HN: What do you use for ML Hosting by quickthrower2 on Wednesday May 03, 2023 @08:46AM (#35801745)

Glad to see this concept being used. I was thinking the other day how odd it is to have IaC then keep the code siloed from the app code. Why doesn’t the app make it’s own infra? I think you have articulated how that would work.

[ Parent | on HN ]