News for hackers. Stuff that matters.

A comment on I Self-Hosted Llama 3.2 with Coolify on My Home Server.

Re:I Self-Hosted Llama 3.2 with Coolify on My Home Server by seungwoolee518 on Wednesday October 16, 2024 @03:14AM (#41856395)
Great post!

However, Do I need to Install CUDA toolkit on host?

I haven't install CUDA toolkit when I use on Containerized platform (like docker)

[ Parent | on HN ]
Re:I Self-Hosted Llama 3.2 with Coolify on My Home Server by thangngoc89 on Wednesday October 16, 2024 @03:18AM (#41856420)
You don't need to install CUDA toolkit on host system.

Nvidia driver + Nvidia container toolkit would do the job. You could check official instructions at [0]

[0] https://docs.nvidia.com/datacenter/cloud-native/container-to...

[ Parent | on HN ]
Re:I Self-Hosted Llama 3.2 with Coolify on My Home Server by seungwoolee518 on Wednesday October 16, 2024 @05:13AM (#41857074)
Thanks, I was bit confused on install a CUDA Toolkit on the Host. (Because I don't install any software except Driver && Toolkit)
[ Parent | on HN ]