Re:A Better Mastodon Client
by ekianjo on Thursday December 21, 2023 @10:49AM
(#38721974)
mistal 7b base model is good enough for summarization. No need to go to the 8x7 one.
[ Parent
| on HN ]
OP here. Yeah, it’s also extremely low-risk. The sensitive part of the algorithm is the embeddings. This part is just making the group human readable.
The reason I didn’t do mistral the first time is because I’m lazy. I gave myself 3-4 hours to get a first pass done, and getting a local model running seemed unnecessarily difficult. It would be a great addition though.