Nvidia's 900 tons of GPU muscle bulks up server market, slims down wallets

Re:Nvidia's 900 tons of GPU muscle bulks up server market, s... by redox99 on Tuesday September 19, 2023 @11:54AM (#37571801)

The 24GB of VRAM of the 3090/4090 (instead of 80GB) is a deal breaker for a lot of stuff. Also 3090 only does 2x nvlink (and 4090 has no nvlink).

But a lot of people (including me) run 3090/4090 rigs at home, or rent them at runpod/vastai to save a buck.

[ Parent | on HN ]

Re:Nvidia's 900 tons of GPU muscle bulks up server market, s... by blopker on Tuesday September 19, 2023 @12:38PM (#37572511)

Since you seem to know and I'm having a rough time finding good info, I'd love to put together a mid range 3090 box to mess around with LLMs. Do you have any pointers to a good build list?

[ Parent | on HN ]

Re:Nvidia's 900 tons of GPU muscle bulks up server market, s... by redox99 on Tuesday September 19, 2023 @01:12PM (#37573049)

> to mess around with LLMs

You mean training or inference?

[ Parent | on HN ]