> Since it is volunteer based, would it be obnoxious to start using some sort of rating system for the readers?

If it was done in the spirit of future improvement, perhaps by ranking them in various dimensions or being prompted with questions such as "How could the narrator improve?" rather than just "Feedback."