Show HN: Klarity – Open-source tool to analyze visual attention patterns in VLM
https://github.com/klara-research/klarity/releases/tag/v0.2
After seeing how difficult it was to debug visual failures in VLMs, we've added visual attention analysis to Klarity. You can now track where your model focuses, measure token-level uncertainty for visual features, and get actionable insights to improve visual grounding.Key features:
- Track visual attention patterns with detailed heatmap visualization - Monitor visual grounding quality and alignment between text and image regions - Identify potential hallucinations and visual reasoning failures - Get concrete suggestions for improving visual prompts and training data
Structured JSON output for visual analysis:
- visual_analysis: {attention_quality, key_regions[], missed_regions[]} - token_attention: array of {word, focused_spot, relevance, uncertainty} - uncertainty_analysis: {problem_spots[], improvement_tips[]} - scores: {visual_grounding, overall_uncertainty, confidence}
Example use cases:
- Debug why your VLM misses important visual details - Optimise visual prompts by understanding attention patterns - Identify which types of images lead to higher uncertainty - Improve visual grounding in your applications
Currently supports Hugging Face transformers (tested with LLaVA models) and Together AI integration for enhanced analysis. Simple to get started with just a few lines of code.
Installation: `pip install git+https://github.com/klara-research/klarity.git`
We're building opensource tools to make generative AI more interpretable and reliable. What challenges are you facing that better debugging tools could help solve?
Links:
- Repo: https://github.com/klara-research/klarity - Our website: [https://klaralabs.com](https://klaralabs.com/) - Discord: https://discord.gg/wCnTRzBE
There are no comments yet.