From the article:

> And, to be honest, it didn’t really matter how much his colleagues protested. As director of the Space Telescope Science Institute, he had a certain amount of Hubble’s time at his personal disposal. “The telescope allocation committee would never have approved such a long, risky project,” he explains. “But as director, I had 10 percent of the telescope time, and I could do what I wanted.”

Also:

> “Scientific discovery requires risk,” Williams says. “And I was at a point in my career where I said, “If it’s that bad, I’ll resign. I‘ll fall on my sword.’”

Well, my opinion (now) is that it wasn't so huge a gamble and completely unknown. The galaxy luminosity function (# galaxies per brightness, basically for those unfamiliar) is a curve they knew the beginnings of through images to date, and it wasn't unreasonable to suppose it would keep going beyond what had been done.

It's not like you would expect galaxies to suddenly stop being found if you took exposures a little deeper. It was a good (and not outrageous) bet. I would liken it to -- if we built a better microscope and looked deeper into a cell, is it likely we'd find nothing new? Probably not.

I'm glad you prefaced this with "(now)" ie: hindsight. There's always a considerable chance that tests of the untested / unproven will remain inconclusive, even if there's some evidence we should find something.

It's extremely risky to allocate a huge amount of finite resources to these unknowns, when there are so many areas we KNOW we can help advance with those finite resources. The fact that he was ready to retire, if the picture came back with barely anything in it, gives a sense of how bad that waste of resources would have been.

Future us might not think it is such a gamble to send a probe to look for life in the oceans of Saturn's moon Enceladus. There's evidence of abiogenesis from deep sea vents on Earth, and Enceladus has very similar conditions. But present us knows there are so many unknowns and possibilities of failure even if something is there. And there are so many other useful ways to use the finite resources of our space program.

It took a lot of courage on Bob Williams' part to explore this.

Except the models they had at the time indicated there should be a lot fewer galaxies than they actually found. Five times fewer. I that’s what the deep field had shown, it would merely have confirmed what people already thought they knew anyway. The reason the actual deep field is so interesting is because it dramatically expanded the scale of the observable universe. If it hadn’t done that, we wouldn’t be talking about it.

10% is a remarkable amount of time. I was astonished when I read that number. I would have expected it to be around 1% at max.

Discretionary time is usually given to projects which otherwise didn't get any time, it's not usually hoarded for the director himself.

I hadn't heard of this concept before, but I find it appealing. It's kind of like an insurance policy against groupthink, or a diversified long bet. More projects should have a small percentage of time dedicated to a drastically different allocation process, with iron-clad protection (like "director's discretion") to prevent manipulation, to help prevent getting stuck.

I find that asymmetric/concentrated power in the appropriate (!) hands can lead to better results for most things. For instance, I'd rather put my money in a company where the CEO has the proper vision and considerable agency over one that is ruled by committee, like most tech titants. I wouldn't advocate for a regression to authoritary regimes, but I think they can stabilize and accelerate progression towards greater objectives as opposed to a segmented and elected political power which rules on a 4 (maybe 8) years "vision" (colored heavily by polls optimization), for example China or ex-Yugoslavia. Linux probably benefited greatly from having Linus. It can obviously go wrong (North Korea, too many kings to count, WeWork, Theranos..).

I think it's a good hedge for NASA to have this policy, they probably selected a good pick in the first place, and if it went wrong they could probably easily replace the position [source needed].

Singapore is a fairly good example of a (mostly) benign dictator - built a modern developed city-state out of what was basically a swamp.

Good example is Google vs Amazon/Apple. Google's culture is very much a democracy of sort inside. Apple/Amazon more top down direction.

The problem, of course, is survivorship bias.

Its similar to Monarchy vs Democracy. A great monarch can get a lot more done than a leader in a democracy. They can also do a lot more harm.

If you want to maximize the potential of a company, you must choose a great CEO and give them near-unlimited power. The problem is that most boards aren't qualified/able to find one, which is why founders that survive the filter to growing into a mega-company are the ones you listed- they are the ones who've already got one.

Tim Cook is doing fine, but he doesn't wield near the influence Jobs did- look how long it took him to finesse Joni Ive out of the company, despite his choices causing massive problems for Apple.

At tech companies, it’s often in the form of hackathons, hackweeks, or 20% time.

Is 20% a real thing anywhere, aside from select google employees?

Googler here. Yes, in my team we have a colleague that is spending one day a week trying to solve a particular problem we have with a completely novel approach. He had an idea after we presented what we are up to in one of our Org summit. Furthermore, it is really a streamlined process, ie you only need your manager approval.

That's not 20% time. That's a project for your primary work area.

When I was at Google (long ago), you were very welcome to do a 20% project. But it was not considered in your performance review.

So you'd better do it outside of your regular job, adding up to a 120% workload.

It is at 3M

The Hubble schedule is planned out roughly a year in advance. If any unexpected astronomical events occur, they use discretionary time to study it.

I adore these images, and deeply respect the reputation risk in getting them.

I also believe that even if we found nothing at all, it would still advance our understanding of the universe as we could confidently say what limits there might be.

Unfortunately people would not have gotten nearly as excited when seeing nothing, and the nay-sayers would have made this very clear.

These things are always as much science as they are politics, so the reputation damage would have been real.

If nothing had been seen, the implications would have been spectacular.

Experiments worth doing are worth doing no matter the result.

The implications aren't automatically "there's nothing out there", more "this was the wrong tool for the job".

True, but I doubt that anyone would have gambled that much Hubble time without working out the statistical sensitivity of the measurement.

Right, I suspect that there was much more behind Bob's deep field study than TFA presents.

More importantly, could he have done the same thing in today’s world and political realities that he did then?

While I agree with you about PR and reputation (sadly), it would be valuable for scientists as a Null value. I would put it on the same level as the Aether experiments.

Yes the point I tried to make was explicitly about the political aspects. I agree that from a scientific point of view, the null value is almost as valuable as what we got now.

I think it's not so much that it was a huge gamble. It was more that it was a sensitive time politically, to make that gamble.

I mean, if they found nothing? Sure, that's still a question Hubble should be trying to answer. But right after such a rocky start, the politically obvious choice would have been to go for easy wins.

The timing was as ballsy as anything else, if not more so.

Wow! The culture sounds very risk averse. I mean 100 hours is not much given how long the telescope remained opertional

HST is in a 96-minute orbit, up to half of which is blocked by the Earth, depending on where in the sky you point it. Getting time on it is an extremely competitive process - the recent cycle had over a thousand proposals, requesting over 24,000 orbits worth of observing time, and they had only 2700 orbits to allocate [1].

This sort of proposal pressure (over 9:1 oversubscription) leads to very conservative proposals, where we request absolute minimums and emphasize that even a non-detection will lead to publishable science in such and such a way.

Getting over 60 orbits to point at a blank field would have been frankly impossible through the regular time allocation process, not because the science wasn't excellent, not because it wouldn't be recognized as excellent, but because it would crowd out other science that could be done with far fewer orbits per project.

The telescope administrators explicitly recognized this problem when they set up the Director's Discretionary Time process (and this is common to all telescopes now) - sometimes you need someone with the authority to just roll the dice on a long shot.

[1] https://www.stsci.edu/contents/newsletters/2019-volume-36-is...

Exactly, this is exactly why I decided not to pursue academia.

Too many times I heard things along the lines of getting "low hanging fruit" to get "published" and that "don't try that you won't ever graduate". Why is graduating and publishing papers and tenure and theses the goal of academia, instead of trying the highest possible risk things that even companies wouldn't want to risk? Why isn't academia about doing the most utterly crazy things possible in the name of advancing science for civilization?

You're absolutely right. What's good for civilization is to advance scientific understanding and knowledge. To take risks, because sometimes they pan out! So go on, pour resources into a thousand crazy ideas. Something will work out eventually.

Yet, what's good for the academic? Each effort costs potentially years of your life. A great discovery will catapult you to the stars, a small one will advance your career, and a negative finding does nothing. You're not alone, you have peers competing for the next round of funding, tenured position, or residency. There are literally never enough to go around, and each will judge applicants in part on their record of results.

I think you're describing a conflict between what's valuable for a system at scale and what's valuable for individual actors within it.

If you want to find nothing without hurting your career, spend your time reproducing other people's results. Lots of the time you will find what they found, which doesn't help your career much. But sometimes you'll find frauds or mistakes, and that will help your career and our civilization quite a lot.

That's very optimistic. More likely you will be disliked by your peers for, in their view, actively trying to demolish their reputation. There's a lot of politics in academia and it's way better to just try to be friends and never say bad words of other peoples works. Live and let live. Is this good for the public and "Science" in capital letters? No. But it's how it is.

Perhaps it is, but you don't have to reproduce the work of your direct colleagues.

Specialties are usually small worlds where everyone knows everyone, reviews, funding grants recommendations flow between them, as do postdocs and graduate students. Stirring up as little water as possible and playing into the hands of the others is incentivized.

This is highly dependent on the discipline and particular field.

For topics with a large number of practicing labs that make use of relatively accessible tooling you might be able to pull off such an approach.

Other fields require customized, incredibly expensive instrumentation. Physics and certain areas of biomedical research are examples of this. It's not uncommon that fewer than ten labs in the entire world will have the capability to perform a particular highly specialized technique.

Even in the largest fields though, it will be your peers reviewing your grant applications at the end of the day.

It’s a question of resource allocation too. To advance science, you want to give more resources and control to people who can potentially make the best discoveries and breakthroughs. How do you find these people? Currently, it seems to be this competition with papers published on the best journals with high citation numbers.

Imagine the world had a single supercomputer, and somebody wanted 100% utilization for those hundred hours to, say, train a neural net to play donkey kong. It's not a huge cost, given the expected lifetime of the resource, but to everybody else waiting in the queue, it would be quite the opportunity cost.

But you don’t sum up all the other things for opportunity cost, you take the single max value. In other words, the opportunity cost doesn’t care about how many other people were waiting, it cares only about how much useful results the best alternative proposal would have gotten with the same amount of hours.

That's an inaccurate picture of resource allocation. 100 hours of a globally unique resource could satisfy 100 1-hour demands. You sum those.

That would be one proposal - give 1 hour to 100 different groups.

What you don’t do is sum the 10000 different requests, nor the 100 requests that also want to use 100 hours each.

This is increasingly common across numerous fields. Certainly in academia, but also in corporate R&D, investing, venture capital, military procurement, etc.

Novel trials are big risks, are institutional (or worse, multi-institutional), and searcch spaces are large, well-picked-over, and huge gains improbabble. Though aversion to novelty is a long-standing behaviour, see "Resistances to the Adoption of Technological Innovations", by Bernhard J. Stern (1937):

https://archive.org/details/technologicaltre1937unitrich/pag...

Markdown https://pastebin.com/raw/Bapu75is