News for hackers. Stuff that matters.
Re:Show HN: Create your own finetuned AI model using Google ... by sneha_tamal on Thursday May 01, 2025 @03:25PM (#43862223)
How do you make sure the fine-tuning process stays effective without the risk of overfitting on specific data?
Re:Show HN: Create your own finetuned AI model using Google ... by manidoraisamy on Thursday May 01, 2025 @04:28PM (#43862987)
Ideally, you want to start small and iterate. With Promptrepo, you can use versioning to compare model outputs across different datasets. In the test UI, we calculate confidence scores using @promptrepo/score [1], which parses OpenAI’s logprobs and shows field-level reliability. Fields with low confidence are highlighted in red, making it easy to catch signs of overfitting or data drift.

[1] https://github.com/ManiDoraisamy/promptrepo-score