News for hackers. Stuff that matters.
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by westurner on Wednesday May 24, 2023 @01:33PM (#36061482)
https://news.ycombinator.com/item?id=35810320 :

> - [ ] dvc, GitHub Actions, GitLab CI, Gitea Actions,: how to add PROV RDF Linked Data metadata to workflows like DVC.org's & container+command-in-YAML approach

https://dvc.org/

https://news.ycombinator.com/item?id=34619424 https://westurner.github.io/hnlog/#comment-34619424 :

- XAI: Explainable AI: https://en.wikipedia.org/wiki/Explainable_artificial_intelli...

- > Right to explanation: https://en.wikipedia.org/wiki/Right_to_explanation

- > A more logged approach with IDK all previous queries in a notebook and their output over time would be more scientific-like and thus closer to "Engineering": https://en.wikipedia.org/wiki/Engineering

Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by shayaks on Wednesday May 24, 2023 @12:49PM (#36060780)
We open-sourced TruLens for LLMs to help evaluate and track your LLM experiments. We also built in special integration with langchain to capture the metadata around your entire chain stack to use with your evaluations. Give it a spin and send us your feedback! We're adding new functionality every day.
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by warthog454 on Wednesday May 24, 2023 @01:24PM (#36061360)
Better LLM app testing is an urgent need when you see the stuff getting put out there.
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by duncanid on Wednesday May 24, 2023 @12:56PM (#36060902)
Smart toolkit to for quick sanity check why developing
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by shayaks on Wednesday May 24, 2023 @02:00PM (#36061824)
Here's a companion blog that explain it works under the hood: https://medium.com/trulens/evaluate-and-track-your-llm-exper...
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by arielmia on Wednesday May 24, 2023 @03:07PM (#36062647)
This is a godsend exactly what i have been searching for
Re:Evaluate and Track Your LLM Experiments: Introducing TruL... by laryb2k on Thursday May 25, 2023 @11:49AM (#36072298)
Amazing and very helpful package!!!