Re:Finetuning Large Language Models
by MacsHeadroom on Saturday April 22, 2023 @04:49PM
(#35670791)
The best option right now is SuperCOT 30B, a LoRA for LLaMA trained on Chain-of-Thought coding questions and answers. You will need at least 24GB of VRAM to load the 4bit GPTQ quantized LoRA merged model* locally. https://huggingface.co/tsumeone/llama-30b-supercot-4bit-128g...
*The merged model contains the LoRA. Applying the LoRA over a "raw" llama model uses more VRAM and does not allow for full context in 24GB of VRAM.
Does anybody know if this is already available on Hugging Face or somewhere else?
I'm using TypeScript, so my idea would be to finetune on all used packages, maybe even all of the documentation as well.
I've created a small app where I create a prompt for an eslint error and let ChatGPT come up with a git diff. I manually feed that back to my application, it eslints and compiles and if there are any errors, a new prompt is generated. This works quite well.
GPT-4 is not available for me in the OpenAI API, so it's quite cumbersome.