News for hackers. Stuff that matters.
Re:Wootric moved ML deployment pipeline from AWS to GCP by ambaragrawal on Tuesday April 14, 2020 @02:10PM (#22869375)
If for a while I eliminate the scaling issue with v0 AWS architecture. Would it be right to say in v0 issues like excessive load times were solved by decoupling models and flask app (e.x. making batch prediction calls to each necessary model for the current request?) rather than v0 architecture itself?

Was it that hard to make the make batch prediction calls to each necessary model for the current request on AWS?

Re:Wootric moved ML deployment pipeline from AWS to GCP by rsmith49 on Tuesday April 14, 2020 @02:21PM (#22869517)
Making the calls to the corresponding model from Flask was actually easier on AWS, since they were loaded into memory. Unfortunately, the scaling issues/excessive load times were big enough of an issue that we had to make the switch, as our number of hosted models continues to grow.
Re:Wootric moved ML deployment pipeline from AWS to GCP by prabhatjha on Tuesday April 14, 2020 @02:19PM (#22869482)
We supported batch api calls in v0. But as those API calls increased a new instanced would get spun up but boot time was longer. To get around it, we would have to keep more instances running all the time which obviously costs more money.