Every time I read this I appreciate its sublimity.

Maybe I'm naive - but I don't get it.

Closing your eyes doesn't make the room empty. And in the same way not programming preconceptions into the neural net doesn't make the preconceptions go away?

I realize explaining a joke or something like this takes away some of the charm (sorry), but would love to get the point :)

Randomly wiring a neural net doesn't remove preconceptions, it just makes it that you don't know them. Similarly, closing your eyes doesn't make a room empty, it just makes it that you can't see what's there. Minsky is pointing out that Sussman's fundamental assumptions on desiring to remove preconceptions is logically flawed.

Gotcha - I read it as Sussman didn't want to program his preconceptions into the network (for which randomness seemed suitable, which is why I was confused). Your explanation makes more sense.

There's an old series of - stories? jokes? called 'unix koans' [1] which always end with a master answering a question in a very unclear but profound-sounding way, then the line 'Upon hearing this, [someone] was enlightened.'

I never found them laugh-out-loud funny myself.

This is probably a reference to those.

[1] http://www.catb.org/~esr/writings/unix-koans/

A Koan (公案) is a concept from Zen Buddhism. Zen made its way into pop-culture (or at least the popular counter-culture movement) in the US in the 1960s via several writers of the Beat Generation. Project MAC at MIT was founded in the early 60s (Gerald Sussman started there in '64 I think)? So a number of faux Koans were in circulation in the AI crowd by about 1970.

Unfortunately because it's delivered as a koan we'll never know whether he's talking about the fact that the random weights determine the nearest local minimum or the hyperparameters.

also unfortunately, a "real koan" often relies on conflating expectations+conditioning versus direct experience, where direct experience is shown to invoke an "impossible" result; teaching that the conditioned, subjective mind does not see fundamental aspects of reality, though it thinks it has the answers.

These technical mimics of that structure echo a time when a lot of people, relatively, were experimenting with disregarding personal subjectivity in favor of direct experience and deeper Truth. In the modern tech versions, that is rarely if ever part of the story?

I think you’ve exactly described the point.

I interpret that as Minsky being... an unpleasurable person, to say the least.

The problem with preconceptions about your parameters is that you might be missing some crazy cool path to your goal, which you might find by randomly exploring your sample space. I remember seeing this same principle in mcmc samplers using uniform priors. Why is this so crazy?

It's predicated on the assumption that a random discovery from a zero-comprehension state is more likely to get you to a goal than an evolution from a state that has at least some correctness.

More generally, it disingenuously disregards the fact that the definition of the problem brings with it an enormous set of preconceptions. Reductio ad absurdum, you should just start training a model on completely random data in search of some unexpected but useful outcome.

Obviously we don't do this; by setting a goal and a context we have already applied constraints, and so this really just devolves into a quantitative argument about the set of initial conditions.

(This is the entire point of the Minsky / Sussman koan.)

> from a zero-comprehension state is more likely to get you to a goal than an evolution from a state that has at least some correctness.

I get that starting from a point with "some correctness" makes sense if you want to use such information (e.g. a standard starting point). However, such information is a preconceived solution to the problem, which might not be that useful after all. The fact is that you indeed might not at all need such information to find an optimal solution to a given problem.

> by setting a goal and a context we have already applied constraints.

I might be missing your point here since the goal and constraints must come from the real world problem to solve which is independent from the method to solve the problem. Unless you're describing p-value hacking your wait out, which is a broader problem.

With exploring, the starting state should only affect which local-maximum you end up in. Therefore you need to make an argument that a random starting state is likely to end up in a higher local-maximum than a non-random starting state.

There is always a starting state; using a random one only means you don't know what it is.

Exactly, but why do so many people seem to have a problem with this? Sounds like a political problem to me instead of a scientific one.