I use to play UT/CS competitively and worked for a startup that licensed our technology to id Software and Riot Games / League of Legends a long time ago.
UT2k4 was an amazing FPS game with a really steep learning curve. It's one of the only FPS games that I refer to as the "basketball" of online gaming. The diversity of movement, weapon tactics, and map control meant a seasoned gamer could really define their own style. But it also meant few people ever transitioned from public servers into competitive play because 1 pro could easily go Godlike and demolish an entire server, making it extremely frustrating and unexciting for casual gamers.
That being said, watching the videos included in this article signaled that these judges had no experience with UT2k4.
In a match with professional gamers, it wouldn't surprise me if those judges thought WE were the bots. 50%+ accuracy was not uncommon with prim shock or lightening gun.
There's a pretty broad area between those two extremes - I'm nowhere near competitive level, but I played (very casually, mainly with friends rather than online) a few hours a week for years, and I definitely understand the game well and can identify good and bad plays, and humans and bots.
I do thoroughly agree, though, that this is nowhere near the level of a true Turing test - but I thought it was still quite interesting!
--
Aside: I think these days with the rise of game spectating and commentary and analysis of matches, more and more non-competitive players are gaining deeper understanding of games they like even if they couldn't necessarily pull it all off themselves.
For a more modern game like Starcraft II for example I think you could find a very large number of non-competitive players that could reliably identify bots from humans.
If the judges were at a competitive level, colour me impressed - but if it was their first time, or even their first week, I'm a little more skeptical. I don't think a novice player would understand the game well enough to judge well. It would be like attempting a traditional Turing test with humans who can't speak English fluently and were raised in a non-English culture: impressive, but no indicator of bots reaching human-like levels.