News for hackers. Stuff that matters.
Re:Stop Evaluating LLMs on Vibes by shayaks on Wednesday June 07, 2023 @08:39AM (#36226031)
Learn more about TruLens here if you're curious: https://medium.com/trulens/evaluate-and-track-your-llm-exper...

Docs and other resources at: https://www.trulens.org/

Re:Stop Evaluating LLMs on Vibes by shayaks on Wednesday June 07, 2023 @08:01AM (#36225724)
Stop deploying LLMs to your users based on vibes. Use TruLens to evaluate your LLM apps
Re:Stop Evaluating LLMs on Vibes by mblemanski on Wednesday June 07, 2023 @10:31AM (#36227245)
Any acceleration in the evaluation process is welcome and much, much needed. Looking forward to checking out TruLens.
Re:Stop Evaluating LLMs on Vibes by shayaks on Wednesday June 07, 2023 @12:24PM (#36229036)
Thanks! Would love to get any feedback.
Re:Stop Evaluating LLMs on Vibes by joshreini on Wednesday June 07, 2023 @10:55AM (#36227577)
Developer on the trulens team here. We just put out a pretty cool new release (0.1.2). Integration with huggingface pipelines in langchain and asynchronous feedback management are just some of the useful updates in this release.

Real evaluations (maybe evaluated by LLMs themselves) instead of just vibe checks is the next big step we need to take as an industry.

Re:Stop Evaluating LLMs on Vibes by vjsingh2793 on Wednesday June 07, 2023 @11:49AM (#36228438)
It's essential to assess language models based on their performance and capabilities rather than subjective measures like "vibes."
Re:Stop Evaluating LLMs on Vibes by duncanid on Wednesday June 07, 2023 @02:40PM (#36231411)
You guys are on fire