Efficient LLM inference solution on Intel GPU

Re:Efficient LLM inference solution on Intel GPU by spearman on Saturday January 20, 2024 @01:34PM (#39070765)

I skimmed the paper but couldn't find it: What API did they use to write their kernels? I would have guessed SYCL since that's what Intel is pushing for GPU programming but I couldn't find any reference to SYCL in the paper.

[ Parent | on HN ]

Re:Efficient LLM inference solution on Intel GPU by spearman on Saturday January 20, 2024 @01:36PM (#39070792)

OK I found it. Looks like they use SYCL (which for some reason they've rebranded to DPC++): https://github.com/intel/intel-extension-for-pytorch/tree/v2...

[ Parent | on HN ]

Re:Efficient LLM inference solution on Intel GPU by mepian on Saturday January 20, 2024 @04:38PM (#39072635)

SYCL is a standard, DPC++ is a particular implementation of this standard.

[ Parent | on HN ]