drob

Sometimes, but I think there are a few problems with this:

1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it.

2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look.

3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.

We should publish some stats. The fix rates on bugs are around 45% baseline, increasing over time as we learn from behavior within a given codebase. The merge rates for the codebase health work we do (e.g. deleting dead code) are very high, at least 90%.

Agreed that domain experts should be making these kinds of decisions, and the right way for an agent mistake to get caught is going to depend a lot on the cost and probability of the mistake. My only point here is that the agent side of the system needs to learn. The software production machine needs to improve over time.

Author here, hello! Happy to expand on how we're thinking about this if any of it is unclear.

We wrote this post as part of a launch, which you can check out here: https://x.com/danlovesproofs/status/2095182189499711759

Should work fine. Ping me [email protected] or @danlovesproofs with your account info so we can look into it?

We've been thinking about this too. We have some ideas. Thanks for the comment, in any case – gave us a lot to chew on.

Other auth providers for sure. We'll be adding shortly.

Using an alternate auth provider won't even prevent you from scanning non-public GitHub code. There's a GitHub OAuth App just for auth (which is what you're seeing here), and a separate GitHub App that you need to install either way to give Detail access to the right repos. We can swap out the former for Google/Okta/pw if you want to avoid this warning. GitHub Apps (the half that manages repo access) have a much finer grained permissions model.

As far as we can tell this is a github-ism, and any OAuth permission is a form of "acting on your behalf": https://dappling.medium.com/a-github-app-would-like-to-act-o...

Hi bflesch, fair point – our About Us page has a lot about what we think and not about... us!

I'm the founder. Previously I was at Heap for nine years. There's a company LinkedIn with the rest of the team: https://www.linkedin.com/company/detail-dev/

We're located in SF. The About Us page lists some of our angel investors at the bottom.

Regarding security in particular, there's a lot more info in our Trust Center: https://trust.detail.dev/

If anything else seems conspicuously missing, please flag. In all likelihood it's omitted without intent.

We've run it on a few firmware repos and gotten good results. A lot of firmware code tends to have really poor type-safety which means lots of low-hanging bugs.

We should be able to handle cross-compilation. Want to try it? Ping me in any direct channel ([email protected] / @danlovesproofs) and we can keep an eye on your repo.

Just github for now, but purely for reasons of plumbing. We'll add gitlab and others.

We support java, c/c++, kotlin, ruby, and swift as well. Did you have something specific in mind?

Github only for now. Out of curiosity, is yours on gitlab? Something else?

We should be able to find something interesting in most codebases, as long as there's some plausible way to build and test the code and the codebase is big enough. (Below ~250 files the results get iffy.) We've just tested it a lot more thoroughly on app backends, because that's what we know best.

Fix is deploying, sorry about that!

Fwiw, the authors never actually claimed this. From their technical report [0]:

> Chai-1 achieves a ligand RMSD success rate of 77%, which is comparable to the 76% achieved by AlphaFold3

[0] https://chaiassets.com/chai-1/paper/technical_report_v1.pdf

On a personal level I can absolutely empathize, but respectfully, I don't see why that should be the state's concern. The goal of IP law should be to promote the creation of good art, not to make sure artists' wishes are respected.

So, for example, theft should be illegal, because a world of unrestricted IP theft might be one in which we would get a lot less art. But allowing Tolkien to block adaptations of his bestseller 14 years after publication was probably not good for art.

Do bad adaptations prevent good adaptations? Lynch's Dune flopped but Villeneuve had a $165m budget.

We can't know the contrapositive but I don't see why giving the author a veto makes good outcomes more likely, especially decades after a book is published.

He owns the IP, but we all lose out from this system.

Art has asymmetric upside – bad art doesn't really harm anyone (usually just gets forgotten) but good art enriches millions of people's lives.

It might have been amazing. It might have been bad-but-interesting. We'll never know!

Zstandard gets us 5.5x compression. The previous ZFS config got us 4.4x compression.

XFS, which we ran on for years before rolling out ZFS, does not compress.

Fixed, thank you for flagging!

Fixed. Thank you again for flagging!

Ah. Yup, it is. Thank you for flagging. Will fix this.