Re:Identifying Rust's collect:<Vec<_>>() memory leak footgun
by vlovich123 on Thursday January 18, 2024 @02:23PM
(#39046354)
Ah right. Vec of vecs and the interior Vec has wasted capacity due to the reduction of the element size. Thanks!
Honestly, if I had 300k Vecs, I would call `shrink_to_fit` even without that optimization.
1: Since the edge list calculation logic is run for one node at a time in sequence, the fact that it temporarily uses 132kb of memory would normally not be a problem at all. However, the new collect() optimization meant that that internal storage was being preserved in the final vec which got inserted into the list of edge lists, thus keeping around that 132kb of obsolete memory forever. Multiply that by 300k nodes and suddenly you’ve leaked the computer’s entire RAM.