timbilt
Google Open Sources Experimental Agent Orchestration Testbed Scion
Kubernetes Resource Optimization Strategies That Work in Production
With public benchmarks we're trusting the labs not to cheat. And it's easy to "cheat" accidentally - they actually need to make a serious effort to not contaminate the training data.
And there's massive incentives for the labs to cheat in order to get the hype going around their launch and justify their massive investments in training. It doesn't have to be the CEO who's directing it. Can even be one/a few researchers who are responsible for a specific area of model performance and are under tremendous pressure to deliver.
This is key.
Public benchmarks are essentially trust-based and the trust just isn't there.
Google's Reverse Acquihire of Windsurf and the Future of AI Developer Tools
Making file encryption fast and secure for teams with advanced key management
Obesity drugs show promise for treating a new ailment: migraine
Slashing CI Costs at Uber
Detecting and Countering Malicious Uses of Claude
Uber's Journey to Ray on Kubernetes: Ray Setup
Why we chose LangGraph to build our coding agent
The Hidden Costs of Men's Social Isolation
Looking back at our Bug Bounty program in 2024
Scaling Up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach
AI is accelerating scientific production, not progress
The Sport of Tuning Quantum Dot Arrays: QDarts Simulator
Improving Search Ranking for Maps
Do We Live in a Special Part of the Universe?
I think automating test coverage would be really useful if you needed to refactor a legacy project — you want to be sure that as you change the code, the existing functionality is preserved. I could imagine running this to generate tests and get to good coverage before starting the refactor.
This seems to be based on the cover agent open source which implements Meta's TestGen-LLM paper. https://www.qodo.ai/blog/we-created-the-first-open-source-im...
After generating each test, it's automatically run — it needs to pass and increase coverage, otherwise it's discarded.
This means you're guaranteed to get working tests that aren't repetitions of existing tests. You just need to do a quick review to check that they aren't doing something strange and they're good to go.