gh-116167: Allow disabling the GIL

Re:gh-116167: Allow disabling the GIL by behnamoh on Monday March 11, 2024 @01:54PM (#39671306)

Not fast enough! I used it to call llama.cpp server but it would crash if requests were "too fast". Calling the llama.cpp server directly solved the issue.

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by ralphist on Monday March 11, 2024 @02:00PM (#39671383)

Did you get "connection reset by peer" when you sent a bit too many requests perchance? I've never found the source of that in my programs. There's no server logging about it, connections are just rejected. None of the docs talk about this.

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by behnamoh on Monday March 11, 2024 @02:11PM (#39671514)

I don't remember the exact error name but the FastAPI server would just freeze.

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by dekhn on Monday March 11, 2024 @03:11PM (#39672162)

a freeze is not a crash.

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by dumdedum on Monday March 11, 2024 @02:55PM (#39672005)

This sounds like a layer 8 problem

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by dekhn on Monday March 11, 2024 @01:57PM (#39671334)

Honestly, that doesn't sound right. I'm curious what you mean by crash thoguh.

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by KaiserPro on Monday March 11, 2024 @05:57PM (#39673701)

Interesting, I've used fastAPI to serve many thousands of requests a second (per process) for a production system. How were you buffering the requests?

[ Parent | on HN ]

Re:gh-116167: Allow disabling the GIL by jononor on Monday March 11, 2024 @09:00PM (#39674952)

You ran FastAPI app proper server like Uvicorn?

[ Parent | on HN ]