How to run Llama 13B with a 6GB graphics card

Re:How to run Llama 13B with a 6GB graphics card by jeron on Sunday May 14, 2023 @09:01PM (#35942828)

imagine an RPG where every NPC is running on its own instance of LLaMa 7B or 13B. It makes the game way more immersive with no predefined script

[ Parent | on HN ]

Re:How to run Llama 13B with a 6GB graphics card by Shocka1 on Wednesday May 17, 2023 @11:33AM (#35976793)

This seems obvious now that I think about it, but it's absolutely fascinating regardless.

[ Parent | on HN ]