nwlotz
nwlotz on Hacker News
The ML research team at Voxel51 just released a paper showing that foundation models rival the accuracy of human annotators in labeling large visual datasets, at several orders of magnitude less time and cost.
We also found that models trained from these labels perform about as well as those trained from human labels when tested against public validation sets. Interestingly, setting a relatively low confidence threshold (0.2 - 0.5) for the auto-generated labels maximized downstream model performance. Very high confidence thresholds often produced worse results due to reduced recall.
The upshot is that zero-shot labeling can replace human annotation in many datasets. The massive cost savings can then be redirected toward training higher-parameter models.
Happy to answer any questions about the research. You can also read this blog we wrote that goes more in depth into the methods and tools we used.
https://link.voxel51.com/HN-VAL-blog/
Posted by
nwlotz on Wednesday June 04, 2025 @11:18AM
from the arxiv.org dept.
There's a fantastic Ray Bradbury short story from 1948 called "Jonah and the Jove-Run" that I hardly see referenced anywhere. It's about Jupiter being the next frontier after colonizing Mars and the complexity of navigating the asteroid belt on supply runs.
It's a great quick read. Though it hardly attempts the sort of scientific justification as in The Three-Body Problem.
"Made with love" is a concept that's subjective but real. You can tell Andor was made with love, while the sequel trilogy looks like it was made with a set of release criteria designed by consultants.
Not to mention I don't even put Andor in the category of a typical Star Wars story. It's just great geopolitical writing. The boardroom scenes were some of my favorites of any show I've ever watched.
One of the best things I was forced to do in high school was read "How to Lie with Statistics" by Darrell Huff. The book's a bit dated and oversimplified in parts, but it gave me a healthy skepticism that served me well in college and beyond.
I think the issues described in this piece, and by other comments, are going to get much worse with the (dis)information overload AI can provide. "Hey AI, plot thing I don't like A with bad outcome B, and scale the axes so they look heavily correlated". Then it's picked up on social media, a clout-chasing public official sees it, and now it's used to make policy.
If even the hyperscalers like OpenAI aren't making a profit, when exactly to companies adopting AI start making money? I wonder if we'll start seeing the hyperscalers start raising prices and squeezing customers when investors finally start expecting a return, and if that will put the breaks on adoption as a result.
I'm curious, why hasn't Valkey picked up corporate sponsors to the degree OpenTofu did when HashiCorp changed Terraform's licensing? I just haven't seen a meaningful level of reaction compared to the community outcry when Hashi changed to BSL.
I've found LLMs to often be a time-suck rather than supercharge my own learning. A huge part of thinking is reconsidering your initial assumptions when you start to struggle in research, mathematical problem solving, programming, whatever it may be. AI makes it really easy to go down a rabbit hole and spend hours filling in details to a question or topic that wasn't quite right to begin with.
Basically analog thinking is still critical, and schools need to teach it. I have no issues with classrooms bringing back the blue exam books and evaluating learning quality that way.
Engineering often supplies the potential energy fueling vision and vice versa. ARPANET was very specifically engineered with the vision of a network that could survive nuclear attack. Then it evolved into the internet, which fueled a new vision of collaboration and discovery for students, hackers, and researchers. Who in turn created systems that became new business models, and the cycle goes on.
I mean code ages quickly, so the value of software must include the skillset needed to support and maintain it. Which is why enterprise software contracts exist, and are expensive. You're not paying for the binary. You're paying for the team supporting it.