Unifying the CUDA Python Ecosystem

Re:Unifying the CUDA Python Ecosystem by sideshowb on Friday April 16, 2021 @04:57PM (#26839003)

Interesting find about the indexing. I just had the opposite experience, swapped from numpy to torch in a project and got 2000x speedup on some indexing and basic maths wrapped in autodiff. And I haven't moved it onto cuda yet.

[ Parent | on HN ]

Re:Unifying the CUDA Python Ecosystem by nuisance-bear on Friday April 16, 2021 @05:22PM (#26839220)

Here's an example that illustrates the phenomenon. If memory serves me right, index latency is superlinear in dimension count.

   import time, torch
   from itertools import product

   N = 100

   ten = torch.randn(N,N,N)
   arr = ten.numpy()

   def indexTimer(val):
       start = time.time()
       for i,j,k in product(range(N), range(N), range(N)):
           x = val[i, j, k]
       end = time.time()
       print('{:.2f}'.format(end-start))

   indexTimer(ten)
   indexTimer(arr)

[ Parent | on HN ]

Re:Unifying the CUDA Python Ecosystem by sideshowb on Friday April 16, 2021 @06:11PM (#26839610)

Ah, I'm guessing it's the loop that kills you. Arrays/tensors are supposed to be used as a whole, I imagine e.g. x=val**2 would be much faster.

I'm indexing one array with another e.g. x=y[z] to pull out all the values I want at once into another array for processing. And not using a python loop for the expensive part.

[ Parent | on HN ]