Re:Octopus v2: On-device language model for super agent
by alwa on Wednesday April 03, 2024 @05:53AM
(#39915344)
Figure 2 is incredible.
With only passing familiarity with the norms in this kind of work, the accuracy rates of all models on this benchmark suite seem suspiciously (and uniformly) high. Is choosing the right intention among “20 vehicle functions” or “20 Android APIs” consistent with an ordinary level of ambition in this kind of research these days?
I just skimmed the paper but this seems to be the crux of it. They map functions to a single token and can then fine-tune models to use the token instead of the function name. This increases accuracy of smaller LLMs and reduces total number of tokens required for prompts and for generations, which is where they get their speed gains from.
The paper is worth a look just to see "Figure (2)"