Go can pass around byte buffers without copying them... a core use case for its core functionality as a network server. The real problem you'd hit with frame processing in Go is that it doesn't give you access to SIMD, nor does it have a nice GPU connection for computation.

I like Go, but I don't really consider myself an "advocate", and part of that is not overpromising to people. If you've got really heavy-duty computational needs, like in science or graphics, it's a bad choice right now, and I expect it to remain so for the forseeable future because it's not a core concern of the devs, and I'm pretty sure they'd view it as dilution of their goals and value proposition. (Correctly, I might add.)