It was important to us to not just translate our Python into C++, but to have C++ that was actually good, C++-style C++. So we hired C++ experts to do all the C++ translations. And same for the other languages.

Now, none of us are C++ experts. So how did we know if the contractors we hired were any good? That's a whoooole blog post for another time.

In short: Noah (an engineer on the team) came up with this clever system where we hired a bunch of people to translate the same code sample (as a paid trial task), then we hired a bunch of people to rate those people's translations (again, as a paid trial task). Then we hired the translator who had the best ratings, and we also hired the /rater/ who was most /thorough/--they do code reviews of the translator's code.

Edit: but it'd make a good one, right?

Some combination of:

- Did they catch stuff that the other reviewers didn't catch?

- Of the stuff they did catch, does it look to us like that's actually stuff that'll help make our code better, or is it just unhelpful pickiness?