Re:MistralLite by Amazon Web Services
by brucethemoose2 on Wednesday November 01, 2023 @01:00PM
(#38101218)
Indeed.
But TheBloke doesn't (yet) do exl2 quantization. That one is almost better done yourself, as the bpw target is arbitrary, and requires profiling text that should probably fit your intended use (though many just use wikitext as a good default).
https://huggingface.co/TheBloke