This is not true. You're losing a crucial property in your translation, namely that the negative numbers are the additive inverse of the positive numbers. In other words, it's essential for electrodynamics that a + (-a) = 0. You could add this requirement to you 'black' and 'white' terminology, but then you've just arrived at negative numbers under a different name.

Don't let the naming confuse you. The name "negative" might be a peculiar, human-specific thing, but the concepts of addition and additive inverses are not.