"in many cases the languages we're dealing with have no writing, so I do agree. A better Olympiad would have at least included a aural-only portion of the exam." Why? A previously unwritten spoken language can always be written with IPA, and giving the test orally would really be a test of people's recognition of sounds they may not have been exposed to even in their phonetics class.

Sign languages are different--there isn't really an IPA-like system for previously unwritten sign languages.