I think 2009 FD is the worrying one. It's incredibly dark, with a surface albedo of just 0.01. That means optical surveys would struggle to find it in deep space, even when it's half a kilometer wide, even with [0] the LSST telescope in the 2020's.

This is why some people want space-based infrared telescopes for NEO surveys [1].

[0] http://arxiv.org/abs/astro-ph/0701506

[1] http://neocam.ipac.caltech.edu/page/whyinfrared