Re:Cortex: Deploy machine learning models in production
by kuu on Friday October 18, 2019 @02:46AM
(#21289191)
How does this work under the hood? Is the model loaded every time it receives a request? Is it run in a docker or a lambda? How does it work after "uploading it" to amazon?
[ Parent
| on HN ]
Re:Cortex: Deploy machine learning models in production
by ospillinger on Friday October 18, 2019 @03:11AM
(#21289285)
Each model is loaded into a Docker container, along with any Python packages and request handling code. The cluster runs on EKS on your AWS account. Cortex takes the declarative configuration from 'cortex.yaml' and creates it every time you run 'cortex deploy' so the containers don’t change unless you run 'cortex deploy' again with updated configuration. This post goes into more detail about some of our design decisions: https://towardsdatascience.com/inference-at-scale-49bc222b3a...