How to Run DeepSeek R1 Distilled Reasoning Models on RyzenAI and Radeon GPUs

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by shosca on Saturday February 01, 2025 @08:24PM (#42904594)

in my case with a 6900xt:

1. sudo pacman -S ollama-rocm

2. ollama serve

3. ollama run deepseek-r1:32b

[ Parent | on HN ]

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by larntz on Saturday February 01, 2025 @08:49PM (#42904805)

Does that entire model fit in gpu memory? How's it run?

I tried running a model larger than ram size and it loads some layers into the gpu but offloads to the cpu also. It's faster than cpu alone for me, but not by a lot.

[ Parent | on HN ]

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by shosca on Saturday February 01, 2025 @09:13PM (#42904971)

you're right, actually noticed gpu clocking up and down with 32b, 14b clocks up fully and actually runs faster

[ Parent | on HN ]

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by heavyset_go on Saturday February 01, 2025 @08:47PM (#42904785)

Nice, last time I tried out ROCm on Arch a few years ago it was a nightmare. Glad to see it's just one package install away these days, assuming you didn't do any setup beforehand.

[ Parent | on HN ]

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by qskousen on Saturday February 01, 2025 @08:59PM (#42904866)

I think you do still have to have the ROCm drivers installed, but it's not very hard to do from AMD's website.

[ Parent | on HN ]

Re:How to Run DeepSeek R1 Distilled Reasoning Models on Ryze... by shosca on Saturday February 01, 2025 @09:13PM (#42904974)

everything from arch repos, well cachyos and arch :)

[ Parent | on HN ]