News for hackers. Stuff that matters.
Re:Indigenous engineers are using AI to preserve their culture by fnordpiglet on Monday February 10, 2025 @11:15PM (#43008893)
Good luck I wish you the best. I think you will almost certainly need to create a LoRA and fine tune an existing model. Is there enough written material available? I think this would be a valuable effort for humanity, as I think the more languages we can model, the more powerful our models will become because they embody different semantic structures with different strengths. (Beyond the obvious benefits of language preservation)
Re:Indigenous engineers are using AI to preserve their culture by antics on Tuesday February 11, 2025 @12:03AM (#43009167)
There is more material than you'd normally find, but it is very hard to even fine-tune at that volume, unfortunately. I think we might be able to bootstrap something like that with a shared corpus of related languages, though.
Re:Indigenous engineers are using AI to preserve their culture by fnordpiglet on Wednesday February 12, 2025 @03:21PM (#43029350)
You can always fine tune with the corpus you have and then try in context on top the fine tuning even if it’s insufficient. Then with that - and perhaps augmenting with RAG against the corpus - you might be able to build a context that’s stable enough in the language that you can generate human mediated synthetic corpuses and other reinforcement to enrich the LORA.