News for hackers. Stuff that matters.

mrciffa

Show HN: MCP Log Reader – Open-source server to analyze MCP logs faster

https://github.com/klara-research/MCP-Analyzer

I've open-sourced MCP Log Reader - an MCP server that helps you analyze and debug MCP server logs. It provides structured access to Claude's logs, making it easy to troubleshoot MCP integrations and understand AI-tool interactions.What MCP Log Reader does: - Multi-platform support for macOS, Windows, and Linux with system-specific log paths - Smart filtering with case-insensitive text...

( Read More... | no comments | on HN )

Show HN: Klarity – Open-source tool to analyze visual attention patterns in VLM

https://github.com/klara-research/klarity/releases/tag/v0.2

After seeing how difficult it was to debug visual failures in VLMs, we've added visual attention analysis to Klarity. You can now track where your model focuses, measure token-level uncertainty for visual features, and get actionable insights to improve visual grounding.Key features:- Track visual attention patterns with detailed heatmap visualization - Monitor visual grounding quality and...

( Read More... | no comments | 4 points | on HN )
It should work with any type of model, obviously longer chain of thoughts will be more difficult to analyse by the evaluation model, because it will have way more reasoning steps to identify and separate. The quality of the outcome depends a lot on the chosen model to give you insights. We tested with Llama3-70B and worked smoothly most of the times.
We are currently giving broad suggestions with an insight model that can be chosen during the setup. We will try to update and improve the suggestion prompt/code to make them more granular with new releases

Show HN: Klarity – OS tool to debug LLM reasoning patterns with entropy analysis

https://github.com/klara-research/klarity/releases/tag/v0.1

After struggling to understand why our reasoning models would sometimes produce flawless reasoning or go completely off track - we updated Klarity to get instant insights into reasoning uncertainty and concrete suggestions for dataset and prompt optimization. Just point it at your model to save testing time.Key new features:- Identify where your model's reasoning goes off track with...

( Read More... | 4 comments | 3 points | on HN )
Yes, reasoning models can potentially be optimized with our uncertainty estimations. We are currently testing the library with DeepSeek R1
Unfortunately LLMs are a gigantic monster to understand, we were considering your same approach with sliding window and we will try to keep the library updated with better and more reliable approaches based on new research papers and our internal tests.
Apache-2.0 is correct one
Exactly! Uncertainty is critical to correctly evaluate LLM performance and we don't need reasoning models to spend thousands of tokens on simple questions
We want to integrate reasoning models as next steps because we see a lot of value in understanding better CoTs behaviour (DeepSeek R1 & Co)
Oh damn, you are right. It's my first opensource project and I didn't thought about it
In the example I'm using the instruction tuned version of Qwen2.5-7B to generate the insights

Show HN: Klarity – Open-source tool to analyze uncertainty/entropy in LLM output

https://github.com/klara-research/klarity

We've open-sourced Klarity - a tool for analyzing uncertainty and decision-making in LLM token generation. It provides structured insights into how models choose tokens and where they show uncertainty.What Klarity does:- Real-time analysis of model uncertainty during generation - Dual analysis combining log probabilities and semantic understanding - Structured JSON output with actionable...

( Read More... | 26 comments | 60 points | on HN )