The main thing is to not pick a saturated color that will clash with anything in the picture/video.

Otherwise, a very light color (white or close to it) or a very dark color (black or close to it) will probably make the picture/video more visible.

FWIW, older users seem to prefer light backgrounds, and younger users seem to prefer dark backgrounds.

Use a neutral gray (this could be black or white). What you want to avoid is framing the image with any "color". Context affects what colors are seen. Knowing a little about something called simultaneous contrast is helpful: http://www.apple.com/pro/color/tools/caponigro/contrast.html

It depends on the effect you want. Since I usually want the details to be highly visible, I usually go for a background the same color but a shade lighter than the major color in the photo. That minimizes the contrast between the photo and its background, bringing out the details within the photo.