idontknowmuch

Viruses are just another "mutagen". No different from UV causing DNA damage in your skin cells, other than the mechanism in which it occurs. The cause for cancer is well-known and, in hindisght, obvious, which is mutation.

The challenge though is mutations can happen in a plethora of ways and their effect is highly dependent on which gene is mutated. There is also the tissue context, e.g. inflammation, spatial structure, etc., that can setup a background for increased mutation. That is why targeted therapies are often the most effective, because they target the general causative feature of a given tumour subtype, the problem is not every protein can be targeted now and each tumour, even within the same subtype has their own unique mutational profile due to the stochasticity of the way mutations occur over repeated rounds of cell division.

And back to viruses, yes they cause cancer because they can mutate DNA. But it's pretty clear, most of the viral "enriched" cancer types are generally in places where transmission is commonplace, e.g. reproductive organs or head/neck.

If you think these types of tools are going to be generating "the most and best research coming out of any lab", then I have to assume you aren't actively doing any sort of research.

LLMs are undeniably great for interactive discussion with content IF you actually are up-to-date with the historical context of a field, the current "state-of-the-art", and have, at least, a subjective opinion on the likely trajectories for future experimentation and innovation.

But, agents, at best, will just regurgitate ideas and experiments that have already been performed (by sampling from a model trained on most existing research literature), and, at worst, inundate the literature with slop that lacks relevant context, and, as a negative to LLMs, pollute future training data. As of now, I am leaning towards "worst" case.

And, just to help with the facts, your last comment is unfortunately quite inaccurate. Science is one of the best government investments. For every $1.00 dollar given to the NIH in the US, $2.56 of economic activity is estimated to be generated. Plus, science isn't merely a public venture. The large tech labs have huge R&D because the output from research can lead to exponential returns on investment.

As noted, I agree on the great strides made in the protein space. However, the over saturation and redundancy in tools and products in this space should make it pretty obvious that selling API calls and compute time for protein binding, annd related tasks, isn’t a viable business beyond the short term.

What tools are "actually working" as of a few years ago? Foundation models, LLMs, computer vision models? Lab automation software and hardware?

If you look at the recent research on ML/AI applications in biology, the majority of work has, for the most part, not provided any tangible benefit for improving the drug discovery pipeline (e.g. clinical trial efficiency, drugs with low ADR/high efficacy).

The only areas showing real benefit have been off-the-shelf LLMs for streamlining informatic work, and protein folding/binding research. But protein structure work is arguably a tiny fraction of the overall cost of bringing a drug to market, and the space is massively oversaturated right now with dozens of startups chasing the same solved problem post-AlphaFold.

Meanwhile, the actual bottlenecks—predicting in vivo efficacy, understanding complex disease mechanisms, navigating clinical trials—remain basically untouched by current ML approaches. The capital seems to be flowing to technically tractable problems rather than commercially important ones.

Maybe you can elaborate on what you're seeing? But from where I'm sitting, most VCs funding bio startups seem to be extrapolating from AI success in other domains without understanding where the real value creation opportunities are in drug discovery and development.

I don’t think the basic usage of references is hard to grok for a beginner. If you aren’t going to mutate data and only access it, then pass a reference. No need for over-complicated semantics when describing it to a new Rust user.

That's completely understandable - leveraging the train set. But what I was trying to say is that the comparison is relative to models that were actually zero-shot and not tuned. It isn't apples to apples, it's apples to orchards.

What's your opinion on the veracity of this benchmark - given o3 was fine-tuned and others were not? Can you give more details on how much data was used to fine-tune o3? It's hard to put this into perspective given this confounder.

Somewhat related — I’d love to hear your thoughts on dex-Lang and Haskell for array programming?

Ahh that's an interesting paper I must of missed that one - thanks for the link. I think another paper that recently got a lot of hype has been the Matroyshka representation learning paper -- essentially training models with different parameters and output embedding sizes at the same time, basically distillation during training rather than post-training (https://arxiv.org/abs/2205.13147).

Afaik, they aren't really trained independently -- for most models, e.g. DINO, etc., the smaller sizes are actually distilled from larger models. It's much easier to generate performant models at smaller size via distillation.

And I'd be curious of the utility of model that scales up and down at inference - if this was the case you'd still need to have storage that is the same as the maximum model size. This would essentially be useless for embedded applications, etc., unless you have heavy quantization - but quantization in a small parameter space would probably make the smaller modes useless. I could see the benefit here in terms of optimizing latency for different applications but maybe you have other ideas.

Given all that, I think training for smaller number of parameters, as noted in OP, would kind of beat out some model that scales at inference time - especially when most people know what kind of application they are aiming to build and the required level of performance.