News for hackers. Stuff that matters.
Re:Identifying Rust's collect:<Vec<_>>() memory leak footgun by WirelessGigabit on Thursday January 18, 2024 @04:04PM (#39047703)
Looking at this example:

    fn dbg_vec<T>(v: &Vec<T>) {
        println!(
            "vec data ptr={:?} len={} cap={}",
            v.as_ptr(),
            v.len(),
            v.capacity()
        );
    }
    
    fn main() {
        {
            let v1 = (0u16..128).map(|i| [i; 1024]).collect::<Vec<_>>();
            dbg_vec(&v1);
            let v2 = v1.into_iter().map(|x| x[0] as u8).collect::<Vec<_>>();
            dbg_vec(&v2);
        }
        {
            let v1 = (0u16..128).map(|i| [i; 1024]).collect::<Vec<_>>();
            dbg_vec(&v1);
            let v2 = v1.into_iter().map(|x| x[0]).collect::<Vec<_>>();
            dbg_vec(&v2);
        }
    }


    # cargo +nightly clean
         Removed 11 files, 7.3MiB total
    
    # cargo +nightly run --release
       Compiling vec-debug v0.1.0 (/home/neo/vec-debug)
        Finished release [optimized] target(s) in 0.17s
         Running `target/release/vec-debug`
    vec data ptr=0x7f47ee162010 len=128 cap=128
    vec data ptr=0x7f47ee121010 len=128 cap=262144
    vec data ptr=0x55c6514f3ba0 len=128 cap=128
    vec data ptr=0x55c6514f3ba0 len=128 cap=131072
    
What happens is that the original's vectors assigned space gets reused. That's it.

It LOOKS like there is more because the capacity inflates. But capacity is written in terms of sizeof(T), not bits

    (0u16..128).map(|i| [i; 1024]).collect::<Vec<_>>()
the capacity (and length) is 128 times an array of 1024 of a u16. We then re-use the same underlying array which has a CAPACITY of 128 times (1024 * 16) and put in a u8 (the cast). So each item went from 1024 * 16 to 8.

So previously we had 128 * 1024 * 16 = 2,097,152 bits.

How many times can we put 8 bits in a capacity of 2,097,152 bits?

262,144

How many times can we put 16 bits in a capacity of 2,097,152 bits?

131,072

Re:Identifying Rust's collect:<Vec<_>>() memory leak footgun by mplanchard on Thursday January 18, 2024 @10:01PM (#39051099)
Thank you for explaining it this way. I’ve been seeing this bit of news off and on all day and knew there was something I wasn’t understanding. This was the key bit