lumpilumpi

Re:Agentic Engineering Patterns on Wednesday March 04, 2026 @03:57AM (#47244855)

My experience is that the first iteration output from a single agent is not what I want to be on the hook for. What squares it for me with "not writing code anymore" is the iterative process to improve outputs:

1) Having review loops between agents (spawn separate "reviewer" agents) and clear tests / eval criteria improved results quite a bit for me. 2) Reviewing manually and giving instructions for improvements is necessary to have code I can own

Re:Evaluating AI agents: Real-world lessons from building agentic syst... on Thursday February 19, 2026 @04:17AM (#47071716)

I get the justification but I found it hard to understand how the actual evaluation at each step is carried out. For example, is there any calibration to some human gold standard involved or is the AI evaluating the AI without calibration/oversight?

Show HN: MCGrad – Fix ML Calibration in Subgroups (Open Source from Meta)

Posted by lumpilumpi on Wednesday February 18, 2026 @02:00PM
from the show-hn dept.

https://github.com/facebookincubator/MCGrad

Hi HN,We're the team at Meta open-sourcing MCGrad. We built this because we found that models often look calibrated on global metrics but fail silently on specific data slices (subgroups).This library provides production-ready implementations of "multicalibration" to detect and fix these local biases.Unlike standard calibration (which looks at the average), MCGrad optimizes for calibration...

( Read More... | no comments | 2 points | on HN )