> Should it be responsibility of Halide to work out the best loop nesting and best use of threads?

I don't know about "should", but it seems to me that it would still be valuable, even if working out the threading and organization.of the data into an array.

> Again, Halide was put aside and we used CUDA for final version, exactly because of inability of Halide to do good work in our case

I didn't say anything about that. I'm not sure why you are restating it.