Functionally speaking, the "features" are a transformation of the image pixel values values,

    F: R^{n x n} => R^d

So `F(image)` is a vector of length `d`, containing `d` derived features from the image. Note that `F` could be a an identity transformation of sorts, which simply flattens the image. In this case, the "features" are simply the image pixel values. However, in general, the feature vector attempts to use a priori information about the problem domain to make things easier down the line.

Now, for binary classification, the classifier is a map,

    C: R^d => {0,1}

To obtain a "good" classifier, one optimizes the parameters of the classifier so that

    C( F(image) ) = label

fairly well, by using database of `( F(image), label )` pairs.

You could a imagine a database of 100px by 100px images of circles and squares. In this case, you might have a hard time obtaining an accurate classifier, if the "features" are simply the image pixel values. On the other hand, a small feature indicating the presence of corners and curvature through some mathematical transformations would likely perform well.