I’ll bet it’s a reference to the Imperial Civil Service Exam[0], which in its later years became notorious in the same ways mentioned in the link. Government officials, who had all passed the exam, were known as Mandarins (at least to the West).

[0]: https://en.wikipedia.org/wiki/Imperial_examination