Re:A comprehensive guide to running Llama 2 locally
by ycombmehair on Tuesday July 25, 2023 @09:01PM
(#36871852)
How would an APU, such as 5700g with up to 128gb of system ram perform when allocating it as vram? Is this a cost effective way of using running this on a budget?
Re:A comprehensive guide to running Llama 2 locally
by NoMoreNicksLeft on Tuesday July 25, 2023 @09:30PM
(#36872065)
Well, 48gb is better than nothing at least. And it has the potential (if we get the build right) to drop a second A6000 card into it with the nvlink module (I think this does allow you to effectively have 96gb) later.
The card's speed will affect your performance, but I don't know enough about different graphics cards to tell you specifics.