Re:How AI tool calling works (40 lines of vanilla JavaScript)
by dvt on Tuesday September 15, 2026 @03:04PM
(#49717236)
> I have seen people actually not get: that LLMs wrap tool calls in special tokens
This isn't necessarily true. I'm working on a local harness that doesn't do this and instead coerces everything to YAML (including tool calls) for better bucketing. Some models are indeed trained on the `<|tool_call>...<tool_call|>` token schema (or something similar—e.g. jinja), but it's vendor-specific and often times inconsistent (so you're constantly fixing calls or going back to the LLM).
Re:How AI tool calling works (40 lines of vanilla JavaScript)
by stephenblum on Tuesday September 15, 2026 @03:49PM
(#49717924)
nice! like HTML tags. I remember seeing this back with meta Llama 2. the <|start_header_id|>assistant<|end_header_id|> style. And [TOOL_CALLS] ... [/TOOL_CALLS] style with Mistral
Re:How AI tool calling works (40 lines of vanilla JavaScript)
by stephenblum on Tuesday September 15, 2026 @03:44PM
(#49717852)
You a are right as The article doesn't mention KV cache. Also, yes the writing is slop, and does not mention the LLM special tokens. Only discusses the use through the OpenAI-style JSON wrapper that allows you to define the schema of a tool call in JSON. For most of the audience, they are looking to build AI agents, and the high-level tool calling interface is what they would be using
It also spends an entire section trying to convey that large tools waste tokens:
but surely that's wrong - it's part of the same conversation, it only gets processed once and then cached. Otherwise doing long conversations would always cost an amount quadratic with length.