News for hackers. Stuff that matters.

A comment on How to run Llama 13B with a 6GB graphics card. Parent comment

Re:How to run Llama 13B with a 6GB graphics card by inferense on Sunday May 14, 2023 @12:33PM (#35939173)
what real use case did you use it for?
[ Parent | on HN ]
Re:How to run Llama 13B with a 6GB graphics card by throwaway1777 on Sunday May 14, 2023 @12:37PM (#35939207)
Making demos to raise investment probably
[ Parent | on HN ]
Re:How to run Llama 13B with a 6GB graphics card by instance on Sunday May 14, 2023 @01:35PM (#35939678)
For instance used it in conjunction with llama-index for knowledge management. Created an index for a whole confluence/jira of a mid-sized company, got good results with GPT, but for LLaMA of this size that use case was too much.
[ Parent | on HN ]
Re:How to run Llama 13B with a 6GB graphics card by sroussey on Sunday May 14, 2023 @05:24PM (#35941454)
Did you try instructor-xl? It ranks highest on huggingface.
[ Parent | on HN ]
Re:How to run Llama 13B with a 6GB graphics card by dzhiurgis on Sunday May 14, 2023 @05:26PM (#35941468)
I'd argue 1k per month for mid-sized company is nothing, but I can understand where you are coming from.
[ Parent | on HN ]