Author of the article here.

I don't think system allocators are clever enough to process and allocate 100-500k of very small objects each minute when Python is performing something very intensive.

It's a pretty standard way to speedup allocation for dynamic languages. Game developers use similar techniques as well.

I have some stats on Python's allocator:

https://rushter.com/blog/python-object-allocation-statistics...

Can confirm for perl. We are doing the very same. It's a huge win.

Differences:

We never free empty pools. Our arenas are just single linked lists, no need for the prev.

Notes:

For a statically compiled perl the biggest win is to avoid arena allocation (mmap) at all. Data and code is made static. That's around 10-20% of the runtime (for shortrunning programs). Also we rarely free at the end. The OS does it much better than free(). Only the mandatory DESTROY calls and FileIO finalizers are executed.

Can Python actually return memory pages back to the OS, e.g. by sbrk() with negative argument?

I'm currently having a problem with this, where I load a large deep learning model into "CPU" memory then move it to the GPU, but I can't get rid of the memory reserved by the process.

I can't answer your exact question, but any large allocations/deallocations should be handled by mmap under the hood and in those cases the memory should be returned.

In your case you should first consider the possibility that there is a pointer to your model's objects that is for some reason not being released. It might simply be that even though you are moving your model the GPU and maybe removing any of your own references, there might be internal references to your model's data that is hidden from you. At least something to consider.

edit: To add to this, I'm now quite sure (though I could be wrong!), that whether python does or does not use sbrk with a negative value is beyond the scope of python. Python is making use of malloc/free under the hood:

https://github.com/python/cpython/blob/master/Objects/obmall...

There's some flexibility for wrapping free in different ways in that file, but it seems like it'll basically always be using free at the core. At least on my system in a debugger I just verified that. So if it's true that python by default uses malloc/free, then the question of whether sbrk with a negative number comes into play is more a question of how your libc implements malloc/free.

Of course I might be wrong, but I think that you should probably stop worrying about it at that level and instead look into object references first as I detailed above.

Yes, it can, by calling the free function.

Which framework do you use for deep learning? It can allocate some object on its own.

Can you give me some stats when using a model and after it's no longer in use and can't be accessible? You can get it by calling the sys._debugmallocstats() function.

The short and simple answer is because it is substantially faster (and has a hugely less memory overhead). It is easy enough to modify the c-python source to try this.

Why it is faster is the article helps cover... some of the tricks for optimizing small object allocation and reuse that c-python performs are not done by the c allocator. As to why THAT is, I can only appeal to your intuition that not all applications allocate memory in the same way so an allocator optimized for Python's behavior has an advantage.

There are a couple of ways of interpreting this question.. but to answer the literal question of why not call malloc/free direcly: See https://docs.python.org/3/c-api/memory.html - it supports custom user-supplied memory allocators, so calling malloc() and free() directly would break this.

I believe this article is taking about small objects allocation (which presumably happens quite frequently in a language like python).

> To reduce overhead for small objects (less than 512 bytes) Python sub-allocates big blocks of memory. Larger objects are routed to standard C allocator. Small object allocator uses three levels of abstraction — arena, pool, and block.

But most system allocators have affordances for small allocations, using techniques such as binning. Is that not good enough for Python?

I don't know if there's a concise summary of the decisions but you can follow the trail of discussions and benchmarks in these links: https://bugs.python.org/issue26249 https://mail.python.org/pipermail/python-dev/2016-February/1...

One explanation could be that the system allocator needs to be thread safe, and this Doren's time with locking. Python is single threaded so perhaps is custom allocator is lock free.

> Python is single threaded

This is inaccurate; especially when considering the thread safety of memory allocation.