How so? Of Steam's users that are running linux and have 'catbot' as part of a username on their system, what proportion do you think are running a cheat bot? 99%? 99.9%?
What level of precision would you say it not 'poor precision'? 99.999%?
It's like treating a disease with antibiotics, eventually the disease builds a tolerance and you need to find a new drug. By using multiple drugs you can hopefully kill it before it can resist any given drug, but in practice tolerance builds regardless of your best efforts.
These cheats are actually big business nowadays, people have a financial incentive to work around them as quickly as possible.
https://github.com/Kr4ken-9/cathook
Golly, hack authors are just so concerned for the well-being of random internet strangers, how dare you doubt their sincerity! /s
So whilst you have a reasonable point, it's not relevant to the point I was making, which is that the precision of the rule (after implementation, and before discovery) probably exceeded 4 nines, which doesn't seem like 'poor precision' to me, without further detail on the requirements.
And BTW I have no opinion about whether this was a good or bad move by Valve. I don't have sufficient context for that, and my comment was intentionally focused to try and understand what the person to whom I replied thought would be acceptable precision.
Plausible or possible? All heuristics have potential false positives (e.g. maybe someone puts the actual catbot binary in the blockchain, and now all Bitcoin users get banned??). I don't see any evidence in the GitHub thread of someone accidentally hit by this. And I see evidence of one person legitimately being hit by this (someone claiming that they are running the bot, just not with the same Steam account).
So I think this heuristic is working properly. (Whether to allow any theoretical risk of a false positive is another discussion, but, Valve is a business, it seems totally reasonable to me for them to risk it.)
Precision is the ratio between correctly-identified cheaters and the total number of people matched by the signature.
Recall is the ratio between correctly-identified cheaters and the total number of cheaters in the system.
In this case, the signature had good recall but poor precision, as those threads describe cases that make it plausible to have a "catbot*" username without cheating.
(I said "had" because the recall rate is going to drop when the string gets changed by the cheat authors. So long-term, the signature is poor on both recall and precision.)
The most problematic issue I see for Valve is not the lack of measurement of precision and recall for their VAC ban signatures, but rather the inability to obsess over their customer experience and react properly when relevant issues in their products are reported.