How to run Llama 13B with a 6GB graphics card

Re:How to run Llama 13B with a 6GB graphics card by jhbadger on Sunday May 14, 2023 @01:08PM (#35939434)

Depends on what "getting stuff done" means. I find 13B models (running on my M1 Mac) useful for playing AI Dungeon-like games -- where you describe the environment and your character and you can play an RPG.

[ Parent | on HN ]

Re:How to run Llama 13B with a 6GB graphics card by fredguth on Sunday May 14, 2023 @01:13PM (#35939476)

GitHub Copilot is (or once was) a 13b model, according to Nat Friedman in the scale.ai interview. (https://youtu.be/lnufceCxwG0)

[ Parent | on HN ]

Re:How to run Llama 13B with a 6GB graphics card by xrd on Wednesday May 24, 2023 @05:33PM (#36064148)

Can you share more about what you are doing or what you've seen others doing?

[ Parent | on HN ]

Re:How to run Llama 13B with a 6GB graphics card by jhbadger on Sunday June 04, 2023 @01:46AM (#36183381)

Just noticed this response, but generally something like "act as a game master of a rpg. The setting is in a medieval style world with magic and my character is a rogue who is good at fast talking and pickpocketing"

Then the LLM generally creates an adventure that I can interact with,

[ Parent | on HN ]