Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on OPT: Open Pre-trained Transformer Language Models. Parent comment
That is use multiple 3090s to load a single model for inference.
I thought that at most you could use two 3090s via NVlink.
Stacking multiple cards would open some real cheap options.
Like a real budget option would be something like a few ancient K80s (24GB version). eBay price was around $200-300 last I checked. .
There are no comments on this comment yet.
That is use multiple 3090s to load a single model for inference.
I thought that at most you could use two 3090s via NVlink.
Stacking multiple cards would open some real cheap options.
Like a real budget option would be something like a few ancient K80s (24GB version). eBay price was around $200-300 last I checked. .