Re:Use pytorch2+cu118 with ADA hardware for 50%+ speedup
by brucethemoose2 on Wednesday July 19, 2023 @04:02PM
(#36792366)
> But I'm sure they have a good reason for this.
CUDA and ROCM make this an intractable problem. Basically there is no way to sanely package everything users need, and the absolutely enormous, cude/rocm versioned pytorch packages with missing libs are already a compromise.
TBH the whole ecosystem is not meant to be for end user inference anyway.
CUDA and ROCM make this an intractable problem. Basically there is no way to sanely package everything users need, and the absolutely enormous, cude/rocm versioned pytorch packages with missing libs are already a compromise.
TBH the whole ecosystem is not meant to be for end user inference anyway.