Unless I'm mistaken (and that's possible, I've changed my opinion twice now), my example outlined above is
- conceivable, and
- has 85% accuracy (85 people correctly identified as liars, 85% x 9900 = 8415 correctly identified as non-liars, thus a total of 85+8415=8500 of 10k total "accurately" identified), and
- still only 5% or 6% of flagged liars are actual liars.
EDIT to add:
And if the system is tweaked as you suggest, to very rarely fail to flag a liar:
- suppose it correctly flags all 100 liars as liars
- suppose accuracy is still 85%, thus 8500 people in total classified correctly
- thus 8400 non-liars flagged correctly, and the remaining 1500 non-liars flagged incorrectly
Now still only 6.25% (100 of 1600) of people flagged as liars are actually liars. Thus, even with the tuning you suggest, this remains.
(Note to self: 1. think 2. write)
You really have to compare precision and recall values to know if the accuracy statement holds true. You could have have 100% precision and low recall and still have 85% accuracy (meaning you could never flag someone as lying and be wrong while missing a bunch of liars and still have 85% accuracy).
but if everything is totally evenly distributed, then 85% accuracy means 85% accuracy and your first statement is correct.
The real issue is that accuracy is only one piece of the puzzle.
https://en.wikipedia.org/wiki/Accuracy_and_precision#In_bina...