HST is in a 96-minute orbit, up to half of which is blocked by the Earth, depending on where in the sky you point it. Getting time on it is an extremely competitive process - the recent cycle had over a thousand proposals, requesting over 24,000 orbits worth of observing time, and they had only 2700 orbits to allocate [1].

This sort of proposal pressure (over 9:1 oversubscription) leads to very conservative proposals, where we request absolute minimums and emphasize that even a non-detection will lead to publishable science in such and such a way.

Getting over 60 orbits to point at a blank field would have been frankly impossible through the regular time allocation process, not because the science wasn't excellent, not because it wouldn't be recognized as excellent, but because it would crowd out other science that could be done with far fewer orbits per project.

The telescope administrators explicitly recognized this problem when they set up the Director's Discretionary Time process (and this is common to all telescopes now) - sometimes you need someone with the authority to just roll the dice on a long shot.

[1] https://www.stsci.edu/contents/newsletters/2019-volume-36-is...

Exactly, this is exactly why I decided not to pursue academia.

Too many times I heard things along the lines of getting "low hanging fruit" to get "published" and that "don't try that you won't ever graduate". Why is graduating and publishing papers and tenure and theses the goal of academia, instead of trying the highest possible risk things that even companies wouldn't want to risk? Why isn't academia about doing the most utterly crazy things possible in the name of advancing science for civilization?

You're absolutely right. What's good for civilization is to advance scientific understanding and knowledge. To take risks, because sometimes they pan out! So go on, pour resources into a thousand crazy ideas. Something will work out eventually.

Yet, what's good for the academic? Each effort costs potentially years of your life. A great discovery will catapult you to the stars, a small one will advance your career, and a negative finding does nothing. You're not alone, you have peers competing for the next round of funding, tenured position, or residency. There are literally never enough to go around, and each will judge applicants in part on their record of results.

I think you're describing a conflict between what's valuable for a system at scale and what's valuable for individual actors within it.

If you want to find nothing without hurting your career, spend your time reproducing other people's results. Lots of the time you will find what they found, which doesn't help your career much. But sometimes you'll find frauds or mistakes, and that will help your career and our civilization quite a lot.

That's very optimistic. More likely you will be disliked by your peers for, in their view, actively trying to demolish their reputation. There's a lot of politics in academia and it's way better to just try to be friends and never say bad words of other peoples works. Live and let live. Is this good for the public and "Science" in capital letters? No. But it's how it is.

Perhaps it is, but you don't have to reproduce the work of your direct colleagues.

Specialties are usually small worlds where everyone knows everyone, reviews, funding grants recommendations flow between them, as do postdocs and graduate students. Stirring up as little water as possible and playing into the hands of the others is incentivized.

This is highly dependent on the discipline and particular field.

For topics with a large number of practicing labs that make use of relatively accessible tooling you might be able to pull off such an approach.

Other fields require customized, incredibly expensive instrumentation. Physics and certain areas of biomedical research are examples of this. It's not uncommon that fewer than ten labs in the entire world will have the capability to perform a particular highly specialized technique.

Even in the largest fields though, it will be your peers reviewing your grant applications at the end of the day.

It’s a question of resource allocation too. To advance science, you want to give more resources and control to people who can potentially make the best discoveries and breakthroughs. How do you find these people? Currently, it seems to be this competition with papers published on the best journals with high citation numbers.

Imagine the world had a single supercomputer, and somebody wanted 100% utilization for those hundred hours to, say, train a neural net to play donkey kong. It's not a huge cost, given the expected lifetime of the resource, but to everybody else waiting in the queue, it would be quite the opportunity cost.

But you don’t sum up all the other things for opportunity cost, you take the single max value. In other words, the opportunity cost doesn’t care about how many other people were waiting, it cares only about how much useful results the best alternative proposal would have gotten with the same amount of hours.

That's an inaccurate picture of resource allocation. 100 hours of a globally unique resource could satisfy 100 1-hour demands. You sum those.

That would be one proposal - give 1 hour to 100 different groups.

What you don’t do is sum the 10000 different requests, nor the 100 requests that also want to use 100 hours each.

This is increasingly common across numerous fields. Certainly in academia, but also in corporate R&D, investing, venture capital, military procurement, etc.

Novel trials are big risks, are institutional (or worse, multi-institutional), and searcch spaces are large, well-picked-over, and huge gains improbabble. Though aversion to novelty is a long-standing behaviour, see "Resistances to the Adoption of Technological Innovations", by Bernhard J. Stern (1937):

https://archive.org/details/technologicaltre1937unitrich/pag...

Markdown https://pastebin.com/raw/Bapu75is