ingridpan

I found this tutorial helpful for getting started with fine-tuning https://www.youtube.com/watch?v=74NSDMvYZ9Y

This guy used gradient.ai and he has a Google Collab to try it

While you're experimenting, worth checking out https://gradient.ai/ -- they're basically the OpenAI API but with llama2

Research: https://arxiv.org/abs/2005.11401

An implementation on Hugging Face: https://huggingface.co/docs/transformers/model_doc/rag

In this post, we explore problems involved in LLM deployment, from GPU shortages to bottlenecks in model performance. These problems have inspired recent developments in distributed training frameworks commonly used to train LLMs, notably ZeRO-Offload. Here we give an overview of ZeRO-Offload, and in future posts we describe its benefits in depth.

RAG is great for pulling some additional knowledge, but if you combine it with fine-tuning (i.e., the LLM 'understands' the domain-specific terminology better) it becomes a lot more effective

https://gradient.ai/ API for inference and fine-tuning open-source LLMs

I think OpenAI is overcharging... because they can

not quite self-hosted but gradient.ai gives you access to llama2 via CLI

https://gradient.ai/ is doing that with llama2