Thanks for the feedback! Right now we have a PR breakdown on the user overview (Completed/Rejected). So that could be a good judge of quality. And I could see pulling in stats from Github (i.e. 'this user has contributed to 25 projects').

But I wouldn't necessarily want to block new developers from being able to contribute to a project. I think it would be best to have anyone able to contribute, but perhaps a flag on the project owner side to see that user's rating.