Using LLaMA with M1 Mac and Python 3.11

Re:Using LLaMA with M1 Mac and Python 3.11 by datadeft on Sunday March 12, 2023 @05:33PM (#35126578)

Not bad with these settings:

    ./main -m ./models/7B/ggml-model-q4_0.bin \
    --top_p 2 --top_k 40 \
    --repeat_penalty 1.176 \
    --temp 0.7 
    -p 'async fn download_url(url: &str)'


    async fn download_url(url: &str) -> io::Result<String> {
      let url = URL(string_value=url);
      if let Some(err) = url.verify() {} // nope, just skip the downloading part
      else match err == None {  // works now
        true => Ok(String::from(match url.open("get")?{
            |res| res.ok().expect_str(&url)?,
            |err: io::Error| Err(io::ErrorKind(uint16_t::MAX as u8))),
            false => Err(io::Error

[ Parent | on HN ]

Re:Using LLaMA with M1 Mac and Python 3.11 by yeeeloit on Sunday March 12, 2023 @06:19PM (#35127034)

What fork are you using?

repeat_penalty is not an option.

[ Parent | on HN ]

Re:Using LLaMA with M1 Mac and Python 3.11 by spion on Sunday March 12, 2023 @11:26PM (#35130606)

It is https://github.com/ggerganov/llama.cpp/blob/master/utils.cpp...

[ Parent | on HN ]

Re:Using LLaMA with M1 Mac and Python 3.11 by beiller on Monday March 13, 2023 @02:14AM (#35131807)

It's a new feature :) Pull latest from master.

[ Parent | on HN ]

Re:Using LLaMA with M1 Mac and Python 3.11 by knodi123 on Monday March 13, 2023 @02:18AM (#35131831)

lol,

    ./main -m ./models/7B/ggml-model-q4_0.bin \
    --top_p 2 --top_k 40 \
    --repeat_penalty 1.176 \
    --temp 0.7
    -p 'To seduce a woman, you first have to'

output:

    import numpy as np
    from scipy.linalg import norm, LinAlgError
    np.random.seed(10)
    x = -2\*norm(LinAlgError())[0]  # error message is too long for command line use
    print x [end of text]

[ Parent | on HN ]