Show HN: OpenLIT – Open-Source LLM Observability with OpenTelemetry

Re:Show HN: OpenLIT – Open-Source LLM Observability with Ope... by LLMtools on Sunday April 28, 2024 @08:35PM (#40193235)

Very cool. QQ: How does this differ from Langtrace (https://www.langtrace.ai) / (https://github.com/Scale3-Labs/langtrace)?

[ Parent | on HN ]

Re:Show HN: OpenLIT – Open-Source LLM Observability with Ope... by Areibman on Sunday April 28, 2024 @10:50PM (#40193907)

Some others have mentioned before, but there are actually several projects working on exactly this:

https://github.com/Arize-ai/openinference

https://github.com/traceloop/openllmetry

https://github.com/Scale3-Labs/langtrace

[ Parent | on HN ]

Re:Show HN: OpenLIT – Open-Source LLM Observability with Ope... by patcher99 on Sunday April 28, 2024 @11:08PM (#40194011)

Yup thats correct, Love as more tools come up! We though tried to follow the actual OpenTelemetry's Semantic Convention for GenAI.

Additionally none of these libraries "then"(Can check again) seemed to send the cost attribute which was what most of our user's mentioned, So we added that.

[ Parent | on HN ]

Re:Show HN: OpenLIT – Open-Source LLM Observability with Ope... by kakaly0403 on Sunday April 28, 2024 @11:57PM (#40194248)

Langtrace core maintainer here. We capture input/output/total tokens for both streaming and non streaming across various LLM providers. For streaming we calculate using tiktoken library as OpenAI does not show that with their outputs. Cost is calculated on the client side as it may keep changing. But all you need is the cost table for the llm vendor and you can get the cost for token usage

[ Parent | on HN ]