News for hackers. Stuff that matters.
Re:Goroutines, Nonblocking I/O, and Memory Usage by rakoo on Thursday January 05, 2017 @04:50PM (#13331748)
Read() being a blocking call, wouldn't some hack like that work ?

    _, err := conn.Read([]byte{})
    if err != io.EOF {
        return err
    }

    // we know there is something to read
    buf := pool.Get().([]byte)
    n, err := conn.Read(buf)

    // process n, err and buf as needed
    // if there is more to read, you may need to loop over conn.Read
    
    // after some inactivity timeout
    buf = buf[:0]
    pool.Put(buf)
Re:Goroutines, Nonblocking I/O, and Memory Usage by zzzcpan on Thursday January 05, 2017 @05:12PM (#13331911)
No, it checks for zero-length buffers and returns immediately. You would have to read at least a byte and copy it into a new buffer.

Won't matter much though, as the memory usage of a single goroutine is quite significant and doubling it by having a buffer preallocated is not something to care about.

Re:Goroutines, Nonblocking I/O, and Memory Usage by masklinn on Thursday January 05, 2017 @05:24PM (#13332013)
> Won't matter much though, as the memory usage of a single goroutine is quite significant

2K as of 1.4 (was 4K before 1.2, 8K in 1.2 and 1.3).

But yeah it's a good point, the intrinsic memory overhead of goroutines means even if buffer pooling worked the memory use of the system would still mostly follow the "naive" estimate.

You may want to message the author to remind them of that, they may not have thought of that concern.

Re:Goroutines, Nonblocking I/O, and Memory Usage by eridius on Thursday January 05, 2017 @05:10PM (#13331904)
Depends on how Read() is implemented. Unless the documentation explicitly says otherwise (and it doesn't look like it does), it would be perfectly reasonable for the method to say "this buffer has a size 0, so I can just return immediately without even touching the fd".
Re:Goroutines, Nonblocking I/O, and Memory Usage by Matthias247 on Thursday January 05, 2017 @05:38PM (#13332104)
Have never tried it with Go and don't know that the contract is there, but the approach in general is valid. Afaik you can use it with Windows IOCP to avoid the occupied buffer problem: Start an async read with a zero byte buffer and when you get the completion notificaton issue a non-blocking read with the then allocated target buffer.