I’m the maintainer for OpenLIT, and like many of you, I’ve always struggled with debugging LLM responses on Ollama and GPT4All—it's possible but with significant effort tbh. So, taking advantage of my role as maintainer, I tackled the problem head-on to make it simpler.
I've developed an OpenTelemetry-native integration within OpenLIT aimed at boosting performance tracking for any LLM hosted on Ollama or GPT4All. Integration takes just one line of code. This integration has notably streamlined my own workflow, allowing monitoring of prompts and responses, including the parameters used, like temperature settings, and how these affect responses, plus tracking token counts.
If you’re encountering similar issues, this could be a really useful: https://docs.openlit.io/latest/integrations/ollama
I'd love to hear how it works for you or discuss any other strategies you've found effective in managing LLMs!
Hey HN, we're super excited to share something we've been working on: OpenLIT. After an engaging preview that some of you might recall, we are now proudly announcing our first stable release!*What's OpenLIT?* Simply put, OpenLIT is an open-source tool designed to make monitoring your Large Language Model (LLM) applications straightforward. It’s built on OpenTelemetry, aiming to reduce the...
https://github.com/openlit/openlit
Hey HN, we're super excited to share something we've been working on: OpenLIT. After an engaging preview that some of you might recall, we are now proudly announcing our first stable release!*What's OpenLIT?* Simply put, OpenLIT is an open-source tool designed to make monitoring your Large Language Model (LLM) applications straightforward. It’s built on OpenTelemetry, aiming to reduce the...
https://github.com/dokulabs/doku
Doku : Open-source platform for evaluating and monitoring LLMs. Integrates with OpenAI, Cohere and Anthropic with stable SDKs in Python and Javascript.