And anyway, the point of my desire is to demonstrate something absolutely convincing, rather than "can spew textual crap at the level of a high school student".
If we get to where you are pointing then we will have passed over a massive gap between today and then, and we're not necessarily that far away from that in time (but still in capabilities).
But likely if and when that time comes everybody that holds this kind of position will move to yet a higher level of attainment required before they'll call it truly intelligent.
So AGI vs AI may not really matter all that much: impact is what matters and impact we already have aplenty.
Done many years ago (2004), without a hint of LLMs or neural networks whatsoever:
https://en.wikipedia.org/wiki/Robot_Scientist
Results significant enough to get a publication in Nature:
https://www.nature.com/articles/nature02236
Obligatory Wired article popularising the result:
Robot Makes Scientific Discovery All by Itself
For the first time, a robotic system has made a novel scientific discovery with virtually no human intellectual input. Scientists designed “Adam” to carry out the entire scientific process on its own: formulating hypotheses, designing and running experiments, analyzing data, and deciding which experiments to run next.
(I work in the field, know those authors, talked to them, elucidated what they actually did, and concluded it was, like many results, simply massively overhyped)
Can you elaborate? Why is it a "bunch of hooey"?
And btw, what do you mean by "overhyped"? Most people on HN haven't even heard of "Adam", or "Eve" (the sequel). I only knew about them because I'm the PhD student of one of the authors. We are in a thread about an open letter urging companies to stop working towards AGI, essentially. In what sense is the poor, forgotten robot scientist "overhyped", compared to that?
For example, the Hershey/Chase and Avery/McCleod experiments convinced the entire biological community that DNA, not protein, was almost certainly the primary molecular structure by which heredity is transferred. The experiments had the advantage of being fairly easy to understand, easy to replicate, and fairly convincing.
There are probably similar simple experiments that can be easily reproduced widely that would resolve any number of interesting questions outstanding in the field. For example, I'd like to see better ways of demonstrating the causal nature of the genome on the heredity of height, or answering a few important open questions in biology.
Right now discovery science is a chaotic, expensive, stochastic process which fails the vast majority of the time and even when it succeeds, usually only makes small incremental discoveries or slightly reduces the ambiguity of experiment's results. Most of the ttime is spent simply mastering boring technical details like how to eliminate variables (Jacob and Monod made their early discoveries in gene regulation because they were just a bit better at maintaining sterile cultures than their competitors, which allowed them to conceive of good if obvious hypotheses quickly, and verify them.