News for hackers. Stuff that matters.

A comment on DeepSpeed Chat: Easy, fast and affordable RLHF training of ChatGPT-like models. Parent comment

Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by nacs on Wednesday April 12, 2023 @07:00PM (#35548226)
> single consumer-grade NVIDIA A6000 GPU with 48GB memory

I wouldn't call an A6000 "consumer-grade" -- it's about $5000 and 99% of consumers that have graphics cards wouldn't have that.

Top of the line consumer grade GPU would be a Nvidia RTX 4090/3090 with 24GB VRAM.

[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by teruakohatu on Wednesday April 12, 2023 @07:01PM (#35548248)
It is solidly a workstation card that is often deployed in data centres. Consumers are far better off with a 4090.
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by freeone3000 on Wednesday April 12, 2023 @07:01PM (#35548240)
“Consumer grade” here means “you can buy it in a store”. (This is not true of DGX devices.)
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by magicalhippo on Wednesday April 12, 2023 @09:13PM (#35549530)
So "off the shelf" rather then.
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by coolspot on Thursday April 13, 2023 @01:12AM (#35551220)
Your local MicroCenter doesn’t stock DGX A100/H100 ???
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by Tepix on Thursday April 13, 2023 @02:02AM (#35551635)
Agreed, but two RTX 3090/4090 should be as capable in this regard (having 2x 24GB).
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by alwayslikethis on Thursday April 13, 2023 @02:18AM (#35551737)
Is it? It might have even more computing power, but are cards now able to share VRAM now? My hands-on experience of all this is from a few years ago and I think it was not possible back then.
[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by Tepix on Thursday April 13, 2023 @05:05AM (#35552833)
DeepSpeed is designed to take care of spreading the work for you.

You can link two 3090s with nvlink to increase bandwidth.

[ Parent | on HN ]
Re:DeepSpeed Chat: Easy, fast and affordable RLHF training o... by mejutoco on Thursday April 13, 2023 @06:12AM (#35553298)
AFAIK the 4090 cannot share ram like that (no nvlink)
[ Parent | on HN ]