Re:RedPajama (Open LLaMA Replication) 3B and 7B models released
by dragonwriter on Saturday May 06, 2023 @12:58PM
(#35843172)
> It seems to me that it’s really hard to get more than 10-14 GB of VRAM without using some sort of hyper expensive cluster.
Both consumer and workstation (the latter may be cheaper per RAM, but with fewer shaders) 16-24 GB GPUs (RTX 3080Ti/3090/4090/A4000/A4500/A5000), including in laptops, are not hard to find (pricey, but not “hyperexpensive clusters”), and its not until you jump above a single 48 GB RTX A6000 that you need a “cluster”.
Both consumer and workstation (the latter may be cheaper per RAM, but with fewer shaders) 16-24 GB GPUs (RTX 3080Ti/3090/4090/A4000/A4500/A5000), including in laptops, are not hard to find (pricey, but not “hyperexpensive clusters”), and its not until you jump above a single 48 GB RTX A6000 that you need a “cluster”.