News for hackers. Stuff that matters.

A comment on LLM that can call multiple tool APIs with one request. Parent comment

Re:LLM that can call multiple tool APIs with one request by _puk on Tuesday June 18, 2024 @02:34AM (#40714689)
> All this is to get back to the point that people want to audit the LLM before running the function because of the unreliability, there is hesitance with good reason.

some people - I think it's quite clear from this thread that not everyone feels the need to.

I'm now thinking requesting the LLM also output its whole prompt to something like a Datadog trace function would be quite useful for review / traceability.

[ Parent | on HN ]
Re:LLM that can call multiple tool APIs with one request by verdverm on Tuesday June 18, 2024 @05:00AM (#40715489)
Most LLM observability tools do this

I'm currently using LangFuse and exploring OpenLit because it integrates with Otel, which you should be able to forward to Datadog iirc their docs

[ Parent | on HN ]
Re:LLM that can call multiple tool APIs with one request by kakaly0403 on Tuesday June 18, 2024 @11:08AM (#40718670)
Checkout Langtrace. It’s also OTEL and integrates with datadog.

https://github.com/Scale3-Labs/langtrace

[ Parent | on HN ]