News for hackers. Stuff that matters.

A comment on Fine-Tuning Llama-2: A Comprehensive Case Study for Tailoring Custom Models. Parent comment

Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by sandGorgon on Friday August 11, 2023 @03:37PM (#37092993)
this is brilliant. could you do a series about how to prepare custom data sets for finetuning. thats the part that a lot of other tutorials skip on. Especially for different goals - like safety, accuracy, etc.
[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by jawerty on Friday August 11, 2023 @04:43PM (#37093758)
Of course I have a few where I web scrape and build a dataset for myself with prefix tokens. I can break that down more on a specific stream about it.
[ Parent | on HN ]
Re:Fine-Tuning Llama-2: A Comprehensive Case Study for Tailo... by sandGorgon on Saturday August 12, 2023 @04:54AM (#37098315)
well not so much as the raw data acquisition (scraping and stuff), but really data prep for finetuning. I'm hearing that each model needs it in a different format - chat finetuning data is different from instruct, etc etc
[ Parent | on HN ]