News for hackers. Stuff that matters.

waelbs

Running Llama2 Models Locally

I get approval from meta, then I downloaded all Llama2 models (7B,7B-chat,13B,13B-chat,70B,70B-chat) locally, when I tried to run the script to test the 7B model using the command that is mentioned on Llama github "torchrun --nproc_per_node 1 example_text_completion.py \ --ckpt_dir llama-2-7b/ \ --tokenizer_path tokenizer.model \ --max_seq_len 128 --max_batch_size 4" I got this...

( Read More... | 2 comments | on HN )