In the original paper by Lamport, Shostak, and Pease, the problem is described this way: "We imagine that several divisions of the Byzantine army are camped outside an enemy city, each division commanded by its own general. The generals can communicate with one another only by messenger. After observing the enemy, they must decide upon a common plan of action. However, some of the generals may be traitors, trying to prevent the loyal generals from reaching agreement."
A common way to describe the participants in such a scenario is simply "honest" and "dishonest". The term "Byzantine" has become used in distributed systems research to refer to situations in which we can't be sure of participants' honesty and intentions. However, it doesn't correctly refer only to the dishonest participants.
How do you decide which one is a "Byzantine" voter? A minority is not necessarily dishonest. If you penalise a minority, at each step at most ceil(n/2-1) of the group will lose their stake, even if they were honest. That sounds overly-harsh.