News for hackers. Stuff that matters.
Re:Memory efficiency of parallel IO operations in Python by jstimpfle on Tuesday March 13, 2018 @09:45AM (#16576007)
Right. And I think a typical stack size is ballpark 8MB, not 8KB, these days.
Re:Memory efficiency of parallel IO operations in Python by scottlamb on Tuesday March 13, 2018 @01:12PM (#16577700)
It varies. 8MiB is believable. That's 8MiB (plus a guard page) of address space. Typically it's divided into 4KiB ("small") pages which are only backed with real RAM once accessed. So maybe only 4KiB of real RAM, depending on what your program is doing...plus whatever bookkeeping the kernel needs, which I think can be as much as 128KiB on Linux, unfortunately. (At least something makes it use that much for me at work, although it might be related to something special I'm doing.)

It's also possible (but not common) to release accessed-but-no-longer-needed pages. Doing this when returning to a thread pool avoids situations where one occasional deep call chain causes all the threads in the pool to eventually become huge.

Re:Memory efficiency of parallel IO operations in Python by Cyph0n on Tuesday March 13, 2018 @11:31AM (#16576922)
I was working with pthreads recently and the default thread stack size is 8M.