fudoshin2596
Re:LLM workflows: Why not just run an experiment?
on Wednesday July 24, 2024 @09:42AM (#41057048)
In so many channels I hear the question, "anyone know if this works" or "what's the impact of X" and I always think to myself, why not just run an experiment? Figured i'd outline a quick workflow that I think scales well for folks building LLM powered products.
Re:Anthropic's Haiku Beats GPT-4 Turbo in Tool Use
on Monday April 08, 2024 @03:59PM (#39973133)
No fine tuning. Looks like he's do raw model capabilities with simple prompt. repo: https://github.com/parea-ai/tool-use-benchmark
Re:Anthropic's Haiku Beats GPT-4 Turbo in Tool Use
on Monday April 08, 2024 @03:57PM (#39973117)
Yea, new tool calling now in public beta is way better. And avoids all of that XML they had before.
LLM evaluation metrics for RAG, chatbots and summarization
Re:Show HN: Unify – Dynamic LLM Benchmarks and SSO for Multi-Vendor De...
on Wednesday February 07, 2024 @09:08AM (#39288733)
The cross region stats are actually quite helpful, this is cool!
Re:How has the ChatGPT model changed from March to June?
on Thursday August 03, 2023 @01:09PM (#36988652)
Interesting. I want to see the GPT 4 results also! But the impact of system message or not was an interesting discovery. Are system messages becoming more or less supported with the new models?
Re:Ask HN: How are you improving your use of LLMs in production?
on Thursday July 20, 2023 @11:28AM (#36801960)
Productionizing LLMs is a Multi headed chimera. From getting the vector db retrieval in order, to proper classification, monitoring, and reliable generations. Still searching for a solution to many of these problems, but https://www.parea.ai helps with the testing, eval, iteration part. Wonder what folks use for the vector db retrieval part?