I think actually the second idea extends quite a bit further than that. The idea is that you enable the agent to interact with an entire collection of laboratory equipment through tool-use. The agent not only generates hypotheses and designs experiments to test them, but then also actually executes the experiments through tool-use and iterates.
This matches what I was told earlier this year when looking into what LLMs could do. I work in public health, and asked staff from a large registry what they'd like to me try. I expected something like generating reports from aggregate data (with some of that automatic exploration mentioned in the article). What they really wanted was:
1. A nice chat bot to answer questions from data submitters about reporting policies.
2. A tool to ingest federal grant announcements and filter down to those the registry could apply for.
#1 is useful because that's a lot of their job. Getting people to send accurate data in the correct format on time is exhausting. #2 helps with everything, because it could mean hiring an additional staff member. That extra person can write reports and apply for grants, clean data, answer calls, and cover when somebody's out sick. Humans are still really useful.
This is half a real insight and half total bullshit. Honestly, what we need robotics and AI for in laboratory science is just old-fashioned standardization and labor-saving.
I see nothing in today's 'deep AI' that addresses these desiderata. And I don't see the current AI strategy of accumulating only existing knowledge as the means to those ends either. Optimization of learning can only learn more facts or do it faster, not think with more creativity or innovation. Memory is but a small part of genius.
Of course not all fields lend themselves as well to this as does materials science.
I think the problems with replication are a separate axis to the problem that there is too much being published for anyone to actually read.
AI in general — never mind LLMs, even the much ones running search engines — help with the content overload.
> This sounds basically of just asking a GPT to suggest ideas for experiments?
I think the car analogy here is that if what you've suggesting was google maps, what the article is suggesting is Level 5 autonomy with no steering wheel and an opaque wall instead of a windscreen.
I have absolutely no idea how hard such a level of automation might be to actually implement, not least because of Moravec's paradox: https://en.wikipedia.org/wiki/Moravec's_paradox
Lots of time can be saved by automating those steps (and many researchers don't enjoy it so their job satisfaction could be increased). Also the resulting output could be improved if the researcher had a well structured summary to use as the foundation of their outline.
Improve search with semantic search (search by concept not keyword) Improve refinement by preprocessing and summarizing Don't print, display clean and concise data. Summarize, cite and display.
This stops short of literature based discovery, you have to bring your own research question.
We've also had some luck finding a gap in existing research. We did a POC where we scraped pubmed and graphed study results by concept. We then used the graphed concepts to explore the conceptual space.
It seems that vitamin D protects against cancer and heart disease. It seems that vitamin D supplementation protects against cancer but not heart disease. Is this because of some previously unknown effect of sun exposure (the primary natural source of vitamin D) or is it just that people with adequate vitamin D go outside a lot more and therefore also get more exercise? Don't know, would love to read the paper if someone studies it ;- )
Yeah, and then the funding agencies summarize the applications by running them through an LLM.
It might be worse than that, not only might ChatGPT uncritically accept authors' claims or reduce the effort to produce low-quality research - given the current state of the art, there seems to be no guarantee that LLMs won't claim things about a paper that was never even said.
> Two areas in particular look promising. The first is “literature-based discovery” (LBD), which involves analysing existing scientific literature, using ChatGPT-style language analysis, to look for new hypotheses, connections or ideas that humans may have missed. LBD is showing promise in identifying new experiments to try—and even suggesting potential research collaborators. This could stimulate interdisciplinary work and foster innovation at the boundaries between fields. LBD systems can also identify “blind spots” in a given field, and even predict future discoveries and who will make them.
I have to question to the value of looking over existing papers given the current replication crises, is using an LLM to review existing papers just going to enable us to do more bad science faster? Does that do anything for us?
> The second area is “robot scientists”, also known as “self-driving labs”. These are robotic systems that use AI to form new hypotheses, based on analysis of existing data and literature, and then test those hypotheses by performing hundreds or thousands of experiments, in fields including systems biology and materials science. Unlike human scientists, robots are less attached to previous results, less driven by bias—and, crucially, easy to replicate. They could scale up experimental research, develop unexpected theories and explore avenues that human investigators might not have considered.
This sounds basically of just asking a GPT to suggest ideas for experiments?
Honestly based on what I understand of science right now the biggest way generative AI could help advanced things is by assisting in the writing of grant applications faster.
EDIT: Relevant XKCD https://xkcd.com/2341/