Re:Octopus v2: On-device language model for super agent
by saltsaman on Wednesday April 03, 2024 @11:33AM
(#39918809)
I can see people training loras this way, which allows for multiple API function calls
[ Parent
| on HN ]
Re:Octopus v2: On-device language model for super agent
by vessenes on Wednesday April 03, 2024 @12:25PM
(#39919439)
Yeah, good idea! I'm not sure if you would be successful mixing LoRA + functional tokens. If you could, that would be great. Then You could ship very light LoRA packs with repositories.
Their LoRA training was I think against their finetuned model, not Gemma-2B directly. But, seems worth playing with -- could be super useful.
Re:Octopus v2: On-device language model for super agent
by AlexChen-NexaAI on Thursday April 04, 2024 @06:52PM
(#39936796)
Yes, you can do that! We have experimented about this idea. The lora training is reported in the paper already, and it works well. I think it would be very interesting.