MLC LLM: Universal LLM Deployment with GPU Acceleration

Posted by crowwork on Friday April 28, 2023 @11:46PM
from the github.com dept.

( 1 comment | 2 points | on HN )

Re:MLC LLM: Universal LLM Deployment with GPU Acceleration by crowwork on Friday April 28, 2023 @11:46PM (#35749830)

Supported platforms include:

- Metal GPUs on iPhone and Intel/ARM MacBooks

- AMD and NVIDIA GPUs via Vulkan on Windows and Linux

- NVIDIA GPUs via CUDA on Windows and Linux

- WebGPU on browsers (through companion project WebLLM).

[ Parent | on HN ]