dmckinno

I should have been a bit more precise. The model is obviously benchmaxxed compared to the western closed models, but it is not compared to the OSS competitors. I think the AA rescoring reflects that as well. Kimi is still in the mix with the tail end of the best models and far ahead of DS, GLM, etc. And the new AA index is obviously wonky in its own way. There is no way Haiku is outperforming Grok 4.7.

I should have been a bit more precise. The model is obviously benchmaxxed compared to the western closed models, but it is not compared to the OSS competitors.

I think the AA rescoring reflects that as well. Kimi is still in the mix with the tail end of the best models and far ahead of DS, GLM, etc.

And the new AA index is obviously wonky in its own way. There is no way Haiku is outperforming Grok 4.7.

Lots of chatter about how this model is benchmaxxed, but didn’t observe this in our testing.

Very strong open model release that is a bit off the frontier but totally amazing for most use-cases.

https://x.com/danielmckinn0n/status/2115213822957117535?a=fO...

OpenAI beat me to the punch in publishing on this, but my results are similar. I hope that these convince you :-)

https://openai.com/index/diagnose-rare-childhood-diseases/

Sharing this both because it is phenomenal work that will bring peace to 18 families and addresses some of the questions raised when I posted my Gamow Labs founding story last week (https://news.ycombinator.com/item?id=48471048).

While I haven't published my results yet (dang OpenAI beat me to it!), they are very similar and I have no doubt that this technology will significantly improve the detection and diagnosis of rare disease, which is a nice win for humanity.

This was short read.

I'm in the process of getting my first batch of long reads, but I am skeptical that this is the "just" what's needed. There is little doubt that long read > short read, but I think that computational techniques for both need to be improved significantly.

There is already some clinical evidence to support my hypothesis. The first clinical long read trial at Kansas City Mercy showed a 10% bump in diagnostic rate, which is great but not fully solving the problem: https://news.childrensmercy.org/unlocking-answers-faster-chi...

Current genomic techniques involve humans using a lot of different software (search, ranking models, visualizations, alignment algorithms, etc.) and synthesizing the results manually into a diagnosis.

Your assumption is correct about my technique. I cloned (and expanded) this workflow into an LLM harness, so the LLM is basically orchestrating a bunch of tools that normally humans would use (and writing the conclusions and doing all the standard LLM stuff).

Thanks for sharing (I am OP). Lots of really interesting computational problems in genetics. The original speech recognition models, HMMs, diffused to genetics ahead of many other fields with which we associate machine learning. Look forward to watching this tech talk.

> Agreed. It does not automatically mean, however, that it can be significantly improved with better variant interpetation or better analysis of the same wgs data in general sense.

I wouldn't say anything is automatic or taken for granted, but it is actually relatively common for more thorough reanalysis to uncover something that the first pass missed. I hinted at this in the post, but the reason that this doesn't happen today is human bandwidth.

A core part of my thesis is that that this highly specialized human bandwidth can be scaled with AI.

It may work. It may not work. But I would feel bad if I didn't give it a try.

> Happy to see it. I wish you all the luck and will be the first one praising your solution if I see convincing results.

Appreciate that! Hopefully, they will come.

Opus-4.8/GPT-5.5 coupled with their respective harnesses will do a good job for more vanilla cases, I realized a custom harness and set of skills and MCP servers became essential as complexity increased (mostly hard SVs). I'll get an eval and more technical post out in a few weeks.

If you are open to chatting about your experience, I'd love to hear from you. I spend a lot of time learning from and supporting other rare disease families these days.

And the anti-natal thing was kind-of joking not joking. I do know lots of people with kids there now, but when my wife first got pregnant, we were alone.

This was not intended to be a technical post (obvious I hope).

I'm planning on getting one out in the next few weeks characterizing the system and how it performed on real clinical use-cases vs. alternatives and existing tools.

The TL;DR is that Gamow Labs is a harness and interface company on top of SOTA LLMs as you suggested, but my harness and interface outperforms the existing thing. While this approach would have earned me the "wrapper company" label last year, I hope the success of OpenEvidence, Harvey, Perplexity, and so on has opened minds with respect to the value here.

It was only working through clinical cases that I realized how much more I needed beyond dropping raw reads into Codex.

It's funny you mention Dr. Kingsmore and Rady.

While I am truly grateful for him and the team for their contributions to neonatal genetics (and hosting me in San Diego for a few days to show me how I could help), Rady was actually the unnamed lab that failed to diagnosis my son.

And this happens all the time. The WGS NICU diagnostic rate is only ~30%, depending on who you ask. Just because people have been working at this for a decade and products exists, doesn't mean it's a solved problem.

I don't know if you read until the end of my post, but I did run a small experiment in collaboration with an academic geneticist and outperformed the first-line clinical labs across the board. My approach, which is essentially Claude Code for genetics, is fundamentally different and novel than how this work is done today and seems to perform much better in early experiments. Time will tell is this generalizes to all clinical work.

I'm planning on publishing evals and benchmarks in the next few weeks, but out-of-the-box systems actually don't do very well for a variety of reasons.

This is kind of like the reverse of the sister comment, which I agree with: https://news.ycombinator.com/item?id=47242699

The general point is that separating PM and eng doesn't make sense any longer. Which subsumes which is an interesting debate.

Your argument that 4.6 Opus makes the engineering skill set useless is totally false and maybe shows you haven't built anything complicated, but it is possible that Opus 5.2 will get there.

I don't remember the exact name, but the one about AI productivity. It should be trivial to find my name from my handle, so just look at my profile.

100%. PMs at startups already wear many hats and AI helps them do that even better.

But to this sister comment's point, I do think that the dedicated PM role will vanish and the classic BigCo PM will need to look a lot more like the startup one.

https://news.ycombinator.com/item?id=47242699

I totally agree (as a PM of ~10 years).

I think that all PMs will need to get onto the engineering, design, or research ladder. We are already seeing companies eliminate the function here and there and I expect the trend to continue.

Communicating a feature with a doc or mock can be really hard. A prototype can make things much clearer to a broad audience.

Thanks! I posted that one almost a year ago and it blew up on LinkedIn of all places but was totally ignored on HN.

You'd be surprised. See this sister comment: https://news.ycombinator.com/item?id=47242372

Weird take. Coding is fun (and has been since before AI). And vibe coding is fun in an entirely different way.

Funny story: I work at Meta and posted a version of this internally in response the bizarre pressure and support for PMs landing prod diffs (the response was very positive FWIW).

Thanks for the flag. VoiceCraft is indeed the best ZS OSS voice cloning tool, despite appearing at the bottom of the TTS arena They have a really easy-to-use gradio demo on their repo if anyone else wants to give it a try.

There is still a big gap between 11Labs and Character.ai and the VoiceCraft voices would not be confused for the real speaker, but this is much closer.