There's a chart in the tiny face paper [.] that compares face size against accuracy. The paper concerns on detecting any faces instead of recognizing a single face though. Seems we can shrink down to 24x19 and still maintain 50% accuracy.

[.] https://openaccess.thecvf.com/content_cvpr_2017/papers/Hu_Fi... Figure 9