dstack (K8s alternative) adds support for AMD accelerators on RunPod

Posted by kaudinya on Wednesday August 21, 2024 @12:21PM
from the dstack.ai dept.

( 8 comments | 23 points | on HN )

Re:dstack (K8s alternative) adds support for AMD accelerator... by abusaidm on Wednesday August 21, 2024 @12:32PM (#41311845)

How is this k8s alternative when the homepage is focusing about running LLM models and AI content in containers rather than being for all workloads?

Maybe it’s k8s alternative for AI/LLM!

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by cheptsov on Wednesday August 21, 2024 @12:39PM (#41311907)

Hi, I’m one of the contributors to dstack! Yes, you’re absolutely right—the title is a bit off. It should be "K8S for AI Teams Only." And yes, there is a lot of focus on LLMs specifically. If you have any other feedback, please share it!

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by abusaidm on Thursday August 22, 2024 @05:50AM (#41318425)

I think framing it as being able to run LLMs with ease and portability across machines and hardware is an amazing sell. Most guides are very technical and detailed that it’s a barrier for many to self host.

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by bj0hn5 on Wednesday August 21, 2024 @02:15PM (#41312739)

Looking forward to trying dStack to deploy NVIDIA NIM and NeMo Microservices.

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by cheptsov on Wednesday August 21, 2024 @02:20PM (#41312786)

That’s on our list! We’re actually planning to expand our examples page with a lot more frameworks! Contributors are very welcome to our repo: https://github.com/dstackai/dstack/tree/master/examples/

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by mdaniel on Wednesday August 21, 2024 @12:33PM (#41311860)

it appears you have borked all the doc links from your readme (e.g.) https://github.com/dstackai/dstack#1-define-configurations

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by cheptsov on Wednesday August 21, 2024 @12:39PM (#41311917)

Oops, thanks for sharing this! Gonna update it shortly.

[ Parent | on HN ]

Re:dstack (K8s alternative) adds support for AMD accelerator... by latchkey on Wednesday August 21, 2024 @12:42PM (#41311947)

We (Hot Aisle) are looking forward to supporting dstack on our MI300x systems as well. They are doing great work.

[ Parent | on HN ]