Re:Perplexity Labs Playground
by swyx on Tuesday December 19, 2023 @03:37PM
(#38701169)
whats the intuition for 2/3 of RAM?
[ Parent
| on HN ]
Re:Perplexity Labs Playground
by avereveard on Tuesday December 19, 2023 @04:02PM
(#38701495)
you need some leftover for holding the context
Because there’s always some overhead during inference plus you don’t want to fill all your available RAM because you risk swapping to disk which will make everything slow to a crawl.