News for hackers. Stuff that matters.
Re:InternLM – new open source 7B LLM by tyfon on Thursday July 06, 2023 @05:09AM (#36613232)
I've only used llama via llama.cpp.

In general I think the python ML stuff is a mess. But I still won't execute code that recommend me to trust arbitrary remote code as the remote code can change at any time, it would be better to wait with the release until it was published to the transformers library or just include it in a clonable repo without the trust_remote_code flag.

It is much better to just be able to clone the code and have it locally so you can verify it once and not trust that it won't download any new code suddenly that you haven't been able to look at.

trust_remote_code means you have no control really, cloning a repo means you control when new code is added yourself.

Re:InternLM – new open source 7B LLM by rfoo on Thursday July 06, 2023 @05:24AM (#36613353)
Yeah, I agree promoting this usage is as bad as promoting `curl | sh` in README.md.

Similar to how you can inspect the content of a `curl | sh` script and then run it, the model is also in a clonable repo, you may just:

   git clone https://huggingface.co/internlm/internlm-7b-chat
and:

    >>> from transformers import AutoTokenizer, AutoModel
    >>> model = AutoModel.from_pretrained("./internlm-chat-7b", device='cuda')
Re:InternLM – new open source 7B LLM by tyfon on Thursday July 06, 2023 @05:51AM (#36613567)
This way is much more palpable for me, thank you for showing :)