Amazing work, I've really wanted to get into knowledge graph generation with LLM's for the last year but haven't found the time. Glad to see someone making good progress on the idea!

How are you going about generating training data?

Lots and lots of manual review of very detailed instructions to "more powerful LLMs" with 2-4 prompts to generate the training data.

Have any references on how you fine tuned?

Sure thing, I used axolotl, and my training parameters were:

sequence_len: 6144 lora_r: 128 lora_alpha: 48 learning_rate: 0.00006 warmup_steps: 600 lr_scheduler: cosine gradient_accumulation_steps: 4 micro_batch_size: 1 num_epochs: 4 optimizer: paged_adamw_32bit flash_attention: true sample_packing: true

Thanks for sharing all these details throughout the thread, Tostino. True open source spirit.

I really like Inkbot! Are you working on a new version? How about one from Yi 34B?

Yeah, I will be soon.

I was busy adding `chat template` support to vLLM recently, so the model (and any others that implement it properly) will work seamlessly with a clone of the OpenAI chat/completions endpoint.

https://github.com/vllm-project/vllm/pull/1756

Now that I have that out of the way, back to model training ;).

Very cool, looking forward to it!

This looks really impressive. Any chance a 7B Inkbot is in the works?

Yeah, i'll be training a Mistral variant soon (and some of the larger, newer models as well).

I had a few dataset issues I know about that I wanted to fix first.

That’s great to hear. Thanks!