Re:Wootric moved ML deployment pipeline from AWS to GCP
by rsmith49 on Tuesday April 14, 2020 @02:21PM
(#22869517)
Making the calls to the corresponding model from Flask was actually easier on AWS, since they were loaded into memory. Unfortunately, the scaling issues/excessive load times were big enough of an issue that we had to make the switch, as our number of hosted models continues to grow.
Re:Wootric moved ML deployment pipeline from AWS to GCP
by prabhatjha on Tuesday April 14, 2020 @02:19PM
(#22869482)
We supported batch api calls in v0. But as those API calls increased a new instanced would get spun up but boot time was longer. To get around it, we would have to keep more instances running all the time which obviously costs more money.
Was it that hard to make the make batch prediction calls to each necessary model for the current request on AWS?