I write an maintain a generic multidimensional array package for Go, and I had come into the same problems as the OP.

The solution was to dictate that the API is copying. But also provide ways to allow the user to manage their own memory.

e.g.

    a = tensor.New(tensor.WithShape(2,2), tensor.Of(Float64))
    b, err := a.Apply(sigmoid)                      // copies
    c, err := a.Apply(sigmoid, tensor.WithReuse(a)) // mutates

Docs: https://godoc.org/gorgonia.org/tensor#Dense.Apply