Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on OpenLLM. Parent comment
> You will need at least 85-100GB of memory to swiftly run inference with Falcon-40B.
So it may be possible with less (swap around method), though not as efficiently and also slower.
There are no comments on this comment yet.
> You will need at least 85-100GB of memory to swiftly run inference with Falcon-40B.
So it may be possible with less (swap around method), though not as efficiently and also slower.