Breaking up with Flask and FastAPI: Why they don’t scale for ML model serving

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by timliu99 on Thursday June 16, 2022 @02:34PM (#31769398)

Wait... so you're tell me FastAPI is slow...

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by ssheng on Thursday June 16, 2022 @03:14PM (#31769795)

FastAPI is great building block but can't expect it to work for model serving out of box.

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by beckingz on Thursday June 16, 2022 @10:45PM (#31773612)

Load your pickle file and it works?

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by beckingz on Thursday June 16, 2022 @10:45PM (#31773614)

I know, pickle is insecure, etc

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by chaoyu_ on Friday June 17, 2022 @11:56AM (#31780097)

Most team don't really concern about the security side of things when using pickle for model deployment, more about performance and resource utilization. It works ok for very light weight models, but doesn't scale.

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by jeremycarter on Thursday June 16, 2022 @05:31PM (#31771222)

Actually it's very slow. Wait until you have 50 concurrent customers hitting it.

[ Parent | on HN ]

Re:Breaking up with Flask and FastAPI: Why they don’t scale ... by nemexis on Thursday June 16, 2022 @06:22PM (#31771733)

That's weird, it is supposed to be faster than alternatives. Do you have some benchmarks to support that critique?

[ Parent | on HN ]