We can probably agree that Feynman's cancer wasn't directly due to his work on the Manhattan Project, but not for the reasons you cite. He died >40 years after the Manhattan Project. Feynman tells several anecdotes which indicate that he and others may have had significant exposure, and that he certainly had many opportunities for exposure. The Manhattan Project was unlike any other science project to follow in that not only was worker safety much less of a concern but the danger was also less well understood than in any successive project. Additionally, all involved persons were encamped together at Los Alamos, Oak Ridge, Hanford, and Argonne, working closely together on a daily basis, milling about each others' labs, sharing other facilities. It is lucky, with the safety regime in place that more people were not killed or harmed.

http://www.orau.org/ptp/pdf/accidentsurvivorslanl.pdf

I'm not sure what you mean; that was exactly my point. His exposure was likely minimal (I can't recall anything suggesting he was close to any criticality assemblage tests), but his exposure level is still more or less irrelevant. It's basically impossible to separate anyone involved in the Manhattan project that didn't die immediately from exposure from expected background population cancer rates.

That's stated pretty explicitly--even for those few most directly exposed in accidents--in the discussion section of the paper you link (which my first linked paper was a followup study to, 20 years later, btw), and it was found to be true in the larger population of nuclear material workers in the second paper (who were, indeed, working in an environment where much more was known about how to control the dangers of what they were working with).

"worker safety much less of a concern but the danger was also less well understood"

Another important point is to people who don't understand the physics, something not understood "must" be feared, therefore the most dangerous part must have been the physics. However I understand at least some of the chemistry and industrial processes using bulk hydrofluoric scare me more than the physics. Aside from the mechanical / explosive hazards, of course (machine plastic explosives to precise shapes on a milling machine for the implosion weapons, are you insane? You're hooking up a zillion blasting caps to .. what?)

Would being a witness to the first blast be a source of radiation?

At the first test explosion of an atomic bomb, Feynman was the only scientist who eschewed protective goggles and watched the blast with unshielded eyes — he wanted to see the explosion clearly, and had researched the danger and confidently concluded that the risk to his vision was negligible.

http://www.nndb.com/people/584/000026506/

IIRC, he sat behind a vehicle's windshield for the benefit of the UV protection it offered. But, no, remember, he died in 1988, more than 40 years later. What type of cancer takes 40 years to show up?

Not very much, no.

Were cancer rates among those witnesses much higher than in the general population? (Note that the goggles really do nothing when it comes to cancer, so that bit is irrelevant.)

The immediate radiation dose from a nuclear explosion is generally low. It's the fallout that gets you, hours or days later. Even then, the risk is acute poisoning, not so much long-term cancer risk.