News for hackers. Stuff that matters.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by stavros on Thursday August 15, 2024 @04:56PM (#41260290)
Have you tried any abliterated models?
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by phren0logy on Thursday August 15, 2024 @05:53PM (#41260796)
Yes, with mixed results.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by ustad on Thursday August 15, 2024 @05:04PM (#41260349)
Any recommendations?
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by chpatrick on Thursday August 15, 2024 @05:13PM (#41260419)
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by reissbaker on Friday August 16, 2024 @02:14AM (#41263600)
Yup, 3.1-70B-Instruct-lorablated is the one I currently recommend too for anti-rejection models — it seems roughly as anti-rejection as the original failspy "abliterated" model, but it works with 128k context since it's based on 3.1 instead of 3 (which only had 8k context). It's currently our second-most popular model on glhf.chat, behind Llama-3.1-405B-Instruct.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by pizza on Thursday August 15, 2024 @05:10PM (#41260399)
failspy’s or mlabonne’s models. Or just look for any model with ‘abliterated’ in the title. Eg try failspy/meta-llama-3-8b-instruct-abliterated-v3 though of course bigger models will probably be better
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by stavros on Thursday August 15, 2024 @05:09PM (#41260380)
No specific ones, but there are some abliteration LoRas for Llama (8B and 70B, I think). Those should be good for what you want.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by rabi_molar on Friday August 16, 2024 @07:26AM (#41265316)
Hadn't heard about the abliteration before, thanks for bringing it up! Here's a HF walkthrough [1] of the concept for anyone else interested in learning more.

[1] https://huggingface.co/blog/mlabonne/abliteration