I assume because 'black people are harder to spot at night time.'

{update} Confirmed:

http://www.etymonline.com/index.php?term=spook

  > The derogatory racial sense of "black person" is
  > attested from 1940s, perhaps from notion of dark
  > skin being difficult to see at night.

Interestingly enough:

  > Black pilots trained at Tuskegee Institute during
  > World War II called themselves the Spookwaffe.

IIRC, Two of those Tuskegee pilots were among the very few Allied pilots to down Nazi jet fighters. From hearing their own accounts on Dogfights, the Tuskegee pilots were exceptionally good at spotting, taking, and hitting opportunistic shots. They were flying early versions of the P-51 Mustang. Downing jet fighters that totally outclass your prop plane because you can recognize an opportunity and take it -- I find that awesome!