Generating a result which looks like some combination of the inputs, has a little bit of 'Wow' factor if you just look at the pictures and don't think too hard about it. But ultimately its not an exercise in which they can be wrong, and since the image returned is always going to be a face, they'll always kind of be right.

An impressive result would be if they trained their system on the the 300 pictures, and then we're reliably able identify which picture the subject was looking at. That would be quantifiable and testable, and I presume the result would expose that this is all nonsense.

So, non-falsifiable...?

"Working with funding from the Yale Provost’s office, Cowen and post doctoral researcher Brice Kuhl, now an assistant professor at New York University, showed six subjects 300 different “training” faces while undergoing fMRI scans. They used the data to create a sort of statistical library of how those brains responded to individual faces. They then showed the six subjects new sets of faces while they were undergoing scans. Taking that fMRI data alone, researchers used their statistical library to reconstruct the faces their subjects were viewing."

So yes, it will always output something like a face. It's more like they are using the FMRI to select among preset options. It's still potentially a great result, but we need more detail than this article provides.

The paper includes a quantitative evaluation, in which they take a set of 30 distractor face images not used elsewhere in the study, and for each of them determine whether the reconstructed face is more similar to it, or to the original face the subject was looking at. On average, the reconstructed face is closer to the correct face than the distractor 62.5% of the time.

So it's better than random, and I think it's pretty cool work, but the quality of the reconstructions is pretty terrible, considering that a randomly chosen distractor will usually be very different from the test face (~half the time opposite sex, frequently different race, different age, etc). For comparison, it would be interesting to evaluate some simple, obviously terrible reconstructions by the same metric. For example, we could "reconstruct" the face as a image that is a single, solid color, the average RGB value of the pixels in the original face. Another "reconstruction" that it seems would very likely do better under this evaluation metric is something like the "race-gender-age" of perp descriptions in the news ("white male in his thirties").

Note the cherry picking of the output for the bottom picture in the Yale press release (forth row in this image).