3d27
We wrote a comprehensive guide on LLM security
How to generate synthetic data using SOTA data evolution methods
How to build your own LLM evaluation framework
Overview of All Major LLM Benchmarks
Re:Show HN: Ragas – the de facto open-source standard for evaluating R...
on Friday March 22, 2024 @03:15AM (#39788104)
Checkout this instead: https://github.com/confident-ai/deepeval
Also has native ragas implementation but supports all models.
I wrote an article about everything I know about LLM metrics
Re:Claude 3 model family
on Monday March 04, 2024 @12:44PM (#39593388)
This is great. I'm also building an LLM evaluation framework with all these benchmarks integrated in one place so anyone can go benchmark these new models on their local setup in under 10 lines of code. Hope someone finds this useful: https://github.com/confident-ai/deepeval
Best practices I learnt from helping health tech enterprise test LLMs
Re:Am I too needy? From a data science perspective
on Monday February 26, 2024 @08:47AM (#39511188)
(found this interesting post on medium, this is not my original work)
Am I too needy? From a data science perspective
Re:AI Infrastructure Landscape
on Monday February 26, 2024 @05:13AM (#39509524)
There's a lot more in the evaluation space, including this one: https://github.com/confident-ai/deepeval
Best Practices for Unit Testing RAG Systems in Prod
Tried Apple's Vision Pros, would not recommend it
Everything I know about LLM evaluation metrics
Google 2024 Layoffs on a rolling-basis
Re:Meta Going All in on GenAI
on Saturday January 20, 2024 @05:06AM (#39066095)
I'm just imagining Jensen Huang laughing in his sleep right now...
Meta Going All in on GenAI
Re:Retrieval Augmented Generation for New Orleans City Council Transpa...
on Wednesday January 03, 2024 @01:54PM (#38858079)
How did you calculate accuracy and bias?
I used QAG to implement an LLM text summarization evals
I found a way to code like Shakespear
I implemented 12+ LLM evaluation metrics so you don't have to
AI Makes Commercial Masterpiece [video]
Re:Show HN: I implemented evals metrics for LLMs that runs locally on ...
on Tuesday December 12, 2023 @02:13AM (#38610566)
The package I built is like a provider for 10+ different evaluation metrics that run both locally on your machine using models from hugging-face but also on the cloud IF you want more functionality.
If you want to evaluate a fine-tuned model, we have integrations with LM Harness and Stanford HELM coming out. If you want to evaluate a RAG application, we have 7+ metrics available for that.
You can also create your custom metrics using our interface!