This project was probably extremely affordable for them and there is value in these tests. They can play around with these different fine-tuning techniques and put them out there for people to mess with while gaining insight into better methodologies for when they spend the big time and cash on larger models.

I'm team lead at Carper. This is correct. It's also just a project one of our engineers did over the course of a few days, so very low risk.

We will swap the base model out for StableLM as soon as we can and iterate from there. We just thought the community would enjoy this research artifact :)

Hopefully RedPajama will change things when (if) they release models based on their independent LLaMa recreation. Until that happens, it might make sense to use the LLaMa model as a stopgap for internal development.

Exactly! It seems pretty clear that someone will reproduce llama and alpaca with a commercially viable model within the next few months to a year at the longest. So if you want to build something based on top of this technology, it makes sense to build it on top of llama/alpaca internally, and then swap it out with something commercially licensed when it’s available. Then you’ve got a huge head start on anyone who is waiting for the stars to align before they start trying to build something at all.