Show HN: Khoj – Chat Offline with Your Second Brain Using Llama 2

Re:Show HN: Khoj – Chat Offline with Your Second Brain Using... by jmorgan on Sunday July 30, 2023 @05:47PM (#36936348)

This is a super cool project. Congrats! If you’re looking at trying different models with one API check out an open-source project a few folks and I have been working on in July in case it’s helpful https://github.com/jmorganca/ollama

Llama 2 gives great answers, even the 7B model. There’s an “uncensored” 7B version as well George Sung has fine-tuned for topics that the default Llama2 model won’t discuss - eg I had trouble having Llama2 review authentication/security code or topics: https://huggingface.co/TheBloke/llama2_7b_chat_uncensored-GG...

From just playing around with it the uncensored model still seems to know where to “draw the line” on sensitive topics but YMMV

If you do end up checking out Ollama you can try it with with this command or there’s an API too (it’s not in the docs yet)

  ollama run llama2-uncensored

[ Parent | on HN ]

Re:Show HN: Khoj – Chat Offline with Your Second Brain Using... by sabakhoj on Sunday July 30, 2023 @08:30PM (#36937570)

This (ollama) is neat! Thanks for the pointers.

Yeah, I ran into a couple of funny edge cases using Llama v2 with my personal notes. For example, if I ever asked it anything remotely personal (as I would with a personal assistant), it would often start telling me that asking for personal data is unethical. I get it, you have to be careful with the open source LLMs, but still a bit funny. It does work with enough coaxing though.

[ Parent | on HN ]

Re:Show HN: Khoj – Chat Offline with Your Second Brain Using... by xcdzvyn on Monday July 31, 2023 @12:32PM (#36945131)

That's actually pretty embarrassing given the product's purpose. I believe there are uncensored LLAMA models out there that might be worth a shot

[ Parent | on HN ]