Anthropic launches Tool Use (function calling)

Re:Anthropic launches Tool Use (function calling) by padolsey on Friday April 05, 2024 @03:31AM (#39939624)

It’s hard to communicate about this stuff. I think people hear ‘tools’ and ‘function calling’ and assume it provides an actual suite of tools or pre-made routines that it calls upon on the Anthropic backend. But nope. It’s just a way of generating a structured schema from a prompt. It’s really crucial work, but just funny how obscured the boring truth is. Also FWIW I experience a tonne more schema adherence if I use XML-like semantic tags rather than JSON. XML is so much more forgiving a format too.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by autonomousErwin on Friday April 05, 2024 @04:21AM (#39939933)

I find this far more useful than a suite of tools or "AI agents" which always work well in a controlled development environment but not so much further than that.

Function calling is a great step towards actually production-izing LLMs and making them extremely robust - I remember when GPT-3 API first came out and I was furiously making sequential calls with complex if/else and try/catch statements and using a couple of Python libraries for the simple reason...I need the output to be a valid JSON. It was surprisingly hard until function calling solved this.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by padolsey on Friday April 05, 2024 @06:13AM (#39940692)

Yeh agreed. Function calling FTW— just need a bit more reliability/(semi-?)-idempotence.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by sdeep27 on Friday April 05, 2024 @08:42PM (#39948973)

Agree. Can really build a strong chain of functionality with this function calling. I have a harder time seeing the use of something like Langchain - seems unnecessary to learn a new bloated API when I can use the powerful tools from the models themselves, and then chain things together myself.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by logicchains on Friday April 05, 2024 @05:26AM (#39940423)

Open LLMs can use grammar-based sampling to guarantee syntactically correct JSON is produced, surprised OpenAI never incorporated anything like that.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by rolisz on Friday April 05, 2024 @07:22AM (#39941029)

my concern with grammar based sampling is that it makes the model dumber: after all, you are forcing it to say something else than what it thought would be best.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by remilouf on Friday April 05, 2024 @01:19PM (#39944856)

Looks like it’s quite the opposite: http://blog.dottxt.co/performance-gsm8k.html

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by CuriouslyC on Friday April 05, 2024 @06:16AM (#39940705)

While some of the downvotes are justified because you're selling this short, I want to point out that your comment about XML is actually valid to a degree. I've found that using XML for prompts lets you annotate specific keywords/phrases and import structure on the prompt which can produce better results.

Getting results back in XML though? That's a terrible idea, you're asking for parsing errors. YML is the best format for getting structured data from LLMs because if there's a parse error you typically only lose the malformed bits.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by epr on Friday April 05, 2024 @07:14AM (#39940978)

XML and other document markup languages are objectively horrible data storage formats. Why is "forgiving" a desired quality in this case?

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by poxrud on Friday April 05, 2024 @08:44AM (#39941701)

It’s much more than just generating structured schema. It also understands user intent and assigns the correct functions to solve a query. So for example if we give it two functions getWeather(city) and getTime(city) and ask “what’s the weather in New York?” It will decide on the correct function to use. It will also know to use both functions if we ask it “what’s the time and weather in New York?”.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by regularfry on Friday April 05, 2024 @09:35AM (#39942192)

I do wonder if a stack-based format would be easier for an LLM. Seems like a better fit for the attention mechanism. My suspicion (without having lifted a finger to check) is that it's the closing tags that make the difference for XML. Go stack-based and you can drop the opening tags, and save the tokens.

[ Parent | on HN ]

Re:Anthropic launches Tool Use (function calling) by sdeep27 on Friday April 05, 2024 @08:40PM (#39948954)

Yes, the 'function calling' naming is unfortunate. It's really structured output that can be fed as input into any functionality elsewhere in your code.

The difference between the structured output of json mode is that the model can choose which set of structured output (matched to various function definitions). Subtle, but pretty cool and powerful.

[ Parent | on HN ]