contemporary343

Yep...

Hahahahaha

Have you ever… been in an actual lab..? There are more disciplines in the world than computer science, you know

Echoing what the other reply to your question said, I don't disagree that much of this post-hoc, but I think the thinking has evolved such that this post-hoc reasoning is now truly believed. I'm primarily speaking of it as an observer of how and where funds are being directed and to what ends: often big, non-university-centered programs to build labs or generate datasets for AI discovery. While some of the ideas may have merit, there is no free lunch unfortunately, as that was money that would have supported individual PIs and their trainees.

One simple way to understand how this is currently manifesting is the administration's severe, and really unprecedented, defunding of basic science at NSF (likely in unconstitutional defiance of Congress' appropriations to specific NSF directorates). Instead the way the funding has been redirected (by the White House OSTP) is based on the assumption that future knowledge will be exclusively AI (sorry SI) generated, so humans not needed.

China, on the other hand, seems to be doubling down on fundamental research, including announcing these changes:

https://www.globaltimes.cn/page/202609/1371583.shtml

and this: https://www.scmp.com/news/china/science/article/3368750/chin...

Finding some new combination or iteration in the literature and running DFT is the kind of thing a senior undergraduate or first year grad student typically does (and typically with Claude anyway these days). (And yes, they'd probably use Quantum Espresso to start, like this writeup and its agent does). They'd probably show it at a weekly lab meeting where it would get ripped apart. And they would not be blasting a preliminary calculation around the world as if they'd made a new discovery.. but hey, we're in a brave new world; maybe they should!

It's an evals platform. The problem is to promote evals in scientific domains you need to actually know something about them. Otherwise you end up with slop like this.

This is what happens when Claude writes it for you (and you don't review it)

They used quantum espresso.. undergrads usually run this in certain classes: https://www.quantum-espresso.org They didn't do any work there.

They ran Quantum Espresso which is ok, but by no means the 'state of the art' for DFT. And in case, any DFT computation has to be taken with a few pounds of grains of salt before getting too excited about it.

No offense to the person writing this (assuming they did at all), but I'm not sure they really understand what they're doing..

I'm sorry but this would be a mediocre paper at best. And if this were a student presenting this for a qualifying exam, you can bet a committee would be ripping them a new one for presenting this with no understanding of what it does.

"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used."

- This, from Tristan Buckmaster's writeup yesterday, indicates to me that there was more than incidental inspiration from Alpoge and Buckmaster.

"I was shown a prompt and told the internal research model had simply been given the problem statement. Levent had been told by Sebastien “very little human input” had been used. This turned out not to be true. Over the course of the call, as members of their team sent Sebastien corrections and details over their internal chat, it emerged that an entire team had been working on the problem, that this was one of a number of things that was tried, that work had started on the unforced problem, that the team first set the model on easier problems, including Euler, that even the prompt that had been shown to me had been written by prompting Codex, and that an insane amount of compute had been used."

One of the interesting threads here that is certainly relevant to the OpenAI writeup is the human role in the process. Buckmaster clearly points out that (exceptional!) mathematicians at OpenAI were certainly involved in correcting and guiding the process - and that their path/strategy was no doubt influenced by Alpoge & Buckmaster's work. It is always in OpenAI's interest to de-emphasize the role of people in the process, as is clearly the case here. Indeed, given sufficient compute and resources, I suspect Buckmaster could have also extended their approach to N-S.

If you haven't read it already, it's really worth digesting the arguments Vannevar Bush made regarding funding basic science in 1945 (Science: The Endless Frontier) which resulted in the NSF as we know it being founded:

https://nsf-gov-resources.nsf.gov/2023-04/EndlessFrontier75t...

"A nation which depends upon others for its new basic scientific knowl- edge will be slow in its industrial progress and weak in its competitive position in world trade, regardless of its mechanical skill."

I think FROs are certainly worth exploring, but if we're diverting vast amounts of funding to existing research infrastructure and talent to funding them, the opportunity costs are huge. This is not well appreciated, but universities are in fact very cheap places to do research compared to any modern alternative one might construct. I suspect FROs will end up being much more expensive just for structural reasons, and will end up re-discovering the university bundle slowly and piece by piece. Moreover, for many types of research they will have to effectively rebuild a range of infrastructure (facilities, equipment, etc.) that already exists at universities throughout the country.

None of this justifies blowing up the extensive basic research infrastructure we have today in pursuit of unproven experiments. Once they're proven, a conversation can be had about how to reconfigure existing assets and new government-funded approaches. But such discussions must include congress, the appropriators of the funds.

I think it's also a way to reduce funding to universities (which are politically disfavored), since other things like arbitrary reductions to indirect costs didn't work. It also defies both congressional will in the appropriations bill (which is directorate-specific) and of course the whole charter and mandate of NSF, from Vannevar Bush's original case for it.

A prescient document that I have come back to many times in my own scientific career. It is striking how little those in power now seem to actually understand what was so clear in 1945:

"To create more jobs we must make new and better and cheaper products. We want plenty of new, vigorous enterprises. But new products and processes are not born full-grown. They are founded on new principles and new conceptions which in turn result from basic scientific research. Basic scientific research is scientific capital. Moreover, we cannot any longer depend upon Europe as a major source of this scientific capital. Clearly, more and better scientific research is one essential to the achievement of our goal of full employment.

How do we increase this scientific capital? First, we must have plenty of men and women trained in science, for upon them depends both the creation of new knowledge and its application to practical purposes. Second, we must strength- en the centers of basic research which are principally the colleges, universities, and research institutes. These institutions provide the environment which is most conducive to the creation of new scientific knowledge and least under pressure for immediate, tangible results. With some notable exceptions, most research in industry and in Government involves application of existing scientific knowledge to practical problems. It is only the colleges, universities, and a few research institutes that devote most of their research efforts to expanding the frontiers of knowledge."

and

"Science Is a Proper Concern of Government

It has been basic United States policy that Government should foster the opening of new frontiers. It opened the seas to clipper ships and furnished land for pioneers. Although these frontiers have more or less disappeared, the frontier of science remains. It is in keeping with the American tradition—one which has made the United States great—that new frontiers shall be made accessible for development by all American citizens.

Moreover, since health, well-being, and security are proper concerns of Government, scientific progress is, and must be, of vital interest to Government. Without scientific progress the national health would deteriorate; without scientific progress we could not hope for improvement in our standard of living or for an increased number of jobs for our citizens; and without scientific progress we could not have maintained our liberties against tyranny."

and

"Freedom of Inquiry Must Be Preserved

The publicly and privately supported colleges, universities, and research institutes are the centers of basic research. They are the wellsprings of knowledge and understanding. As long as they are vigorous and healthy and their scientists are free to pursue the truth wherever it may lead, there will be a flow of new scientific knowledge to those who can apply it to practical problems in Government, in industry, or elsewhere."

Actually overheads for many universities were sometimes higher in the late 1990s (and there were some minor scandals associated with this). And remind me again, what fraction of our GDP is indirect costs to universities? (< 0.1%). And what are the benefits? Well, indirect costs are how the U.S. government builds up a distributed network of scientific and technical infrastructure and capacity. This capacity serves the national interest.

If you think you're going to help debt by cutting indirect costs and crippling university research permanently, may I introduce you to the foundational notions of a knowledge economy and how fundamental advances feed into technology developments that increase productivity and thus GDP. Permanently reducing growth is another way of making debt servicing worse.

In light of all of their actions, I really, genuinely do not understand what DOGE team members' mental model of the world is and what they view their purpose as. None of these actions align with any publicly stated goals, or even the privately reported ones. All I see is a destruction of state capacity that has already left America significantly weakened and less capable of responding to threats.

Definitely seems like a drug problem to me. He seems strung out of his mind if this article's sourcing is correct: https://www.wired.com/story/katie-stephen-miller-elon-musk-t...

“We will not use our skills as technologists to compromise core government systems, jeopardize Americans’ sensitive data, or dismantle critical public services,” they wrote. “We will not lend our expertise to carry out or legitimize DOGE’s actions.”

“‘Move fast and break things’ may be acceptable to someone who owns a business and owns the risk. And if things don’t go well, the damage is compartmentalized. But when you break things in government, you’re breaking things that belong to people who didn’t sign up for that,” said Cordell Schachter, who until last month was the chief information officer at the U.S. Department of Transportation.